Intelligent transportation systems (ITS) are integrated systems that use advanced computer and communication technologies in road transportation, including vehicles, users, and infrastructure, and in traffic and mobility management, as well as for interfaces with other modes of transportation. There are challenging dynamic decision problems and critical questions that readily stand to benefit tremendously from artificial intelligence (AI), for example:

  • How should vehicles and passengers be matched in an optimal way to maximize the efficiency of a mobility-on-demand system?
  • How do we rebalance/reposition a fleet of vehicles to maximize overall system utilization?
  • How can traffic signals adapt intelligently to the dynamic traffic patterns?
  • How does an autonomous vehicle respond to human driver behavior in a mixed autonomy traffic?
  • What can be done in particular to generate safe control policies in an ITS?
A lot of these problems have a common characteristic that a sequence of decisions are to be made while we care about some cumulative objectives over a length of horizon. Reinforcement learning (RL) is a machine learning paradigm that trains an agent to learn to take optimal actions (as measured by the total cumulative reward achieved) in an environment through interactions with it and getting feedback signals. It is thus a class of optimization methods for solving sequential decision-making problems. Thanks to the rapid advancement in deep learning research and computing capabilities, the integration of deep neural networks and RL has generated explosive progress in the latter for solving complex large-scale learning problems, attracting huge amount of renewed interests in the recent years. The combination of deep learning and RL has even been considered as a path to general AI. It presents a tremendous potential to solve some hard problems in transportation in an unprecedented way and to have profound positive impact on the daily lives of our community. RL for ITS is an emerging interdisciplinary area that is undergoing rapid development and still require further understanding on a number of issues, e.g., dynamic environments, run-time performance, and safe deployment. The main goal of this workshop is to bring together researchers and practitioners from both the RL and transportation communities to answer the following questions:
  • What are the recent advances in the state-of-the-art of RL for ITS?
  • What are the new application areas in transportation that can be benefited by RL?
  • What are the successful stories in real-world applications of RL to ITS?
  • What are the challenges and critical issues that may have significant impact on a successful application?
Answering these questions is critical to unleash the potential of the core intelligence in ITS beyond classical machine learning. We hope the participants will leave the workshop with a better idea about the current status of RL for ITS, about what is working well and what is not, thus providing guidance on directions of future research in this exciting area.


The workshop Zoom meeting can be accessed through the IJCAI virtual conference platform .

Walk the avatar to meeting room Yellow 3 and press X upon entering.

Youtube Live broadcast

All times are in Eastern Time on Aug 21, 2021.
Time Activity
10:15 – 10:30 Opening remarks
10:30 – 11:30 Invited Talk: Javier Alonso-Mora (Delft University of Technology)
11:30 – 12:30 Invited Talk: Jun Luo (Huawei Noah’s Ark Lab Canada)
12:30 – 13:15 Spotlight:

  • Bounding the Inefficiency of Route Control in Intelligent Transport Systems
  • Periodic-GP: Learning Periodic World with Gaussian Process Bandits
  • A Review of Real-time Railway and Metro Rescheduling Models using Learning Algorithms
13:15 – 14:30 Break
14:30 – 15:30 Invited Talk: Alexandre Bayen (UC Berkeley)
15:30 – 16:15 Spotlight:

  • Pattern Transfer Learning for Reinforcement Learning in Order Dispatching
  • A Linear-Programming and Prediction based Lookahead Policy for Vehicle Repositioning in Ride-Hailing Systems
  • UAVs for Search and Rescue: A Reinforcement Learning Approach
19:15 – 19:45 Spotlight:

  • PRGLight: A novel traffic light control framework with Pressure-based-Reinforcement Learning and Graph Neural Network
  • Towards a Very Large Scale Traffic Simulator for Multi-Agent Reinforcement Learning Testbeds
20:00 – 21:00 Invited Talk: Hai Yang (The Hong Kong University of Science and Technology)
21:00 – 22:00 Invited Talk: Pradeep Varakantham (Singapore Management University)
22:00 – 23:00 Invited Talk: Jessie Li (Yunqi Academy of Engineering)

Call for Papers


We invite paper submission with a focus that aligns with the goals of this workshop. Our topics of interest span over planning, control, and decision problems in mobility-on-demand, smart and automated transportation systems and beyond, including but not limited to

  • ridesharing marketplace: order matching, driver/vehicle repositioning, carpooling, pricing,
  • traffic control, smart infrastructure,
  • simulation for traffic, taxi, or ridesharing,
  • mixed autonomy traffic and autonomous driving,
  • truck fleet management,
  • vehicular robotics: planning, navigation and routing,
  • managing uncertainty in transportation systems,
  • RL with safety constraints for transportation systems.


We look for innovative applications of RL and highly related methods, as well as the employment of RL techniques in conjunction with traditional approaches for ITS.

Submission portal:

Paper format: Papers submitted to RL4ITS Workshop must be formatted according to the IJCAI-21 guidelines (link above). Submissions must be self-contained. Authors are required to submit their electronic papers in PDF format.

Paper length: Papers must be no longer than 7 pages in total: 6 pages for the body of the paper (including all figures/tables), plus up to 1 additional page with references that do not fit within the six body pages. Authors may submit up to 50MB of supplementary material, such as appendices, proofs, derivations, data, or source code; all supplementary material must be in PDF or ZIP format.

The reviewing process is double-blind. At least one author of each accepted paper is required to attend the workshop to present the work. Authors will be required to agree to this requirement at the time of submission.

The workshop is non-archival, and there will not be formal proceedings, although papers will be available on the workshop website. Papers that are under review at another conference or journal are acceptable for submission at this workshop, but we will not accept papers that have already been accepted or published at a venue with formal proceedings (including IJCAI 2021). The submitted papers will be peer-reviewed by at least two PC members. A Best Paper Award will be presented to the best submitted paper as voted by the reviewers. All papers are to be presented at the poster sessions. A select group of papers will be given the opportunity to present at the spotlight sessions during the workshop. We also plan to invite the spotlight papers to expand and submit to a reputable journal in computer science or transportation.

Important Dates

Submission due: May 12, 2021 (23:59 Pacific Time)
Acceptance notification: May 25, 2021 (23:59 Pacific Time)
Camera-ready due: June 22, 2021 (23:59 Pacific Time)


For questions on submission and the workshop, please send email to

Invited Speakers (in alphabetical order)

Javier Alonso-Mora

Associate Professor, Delft University of Technology

Alexandre Bayen

Professor, UC Berkeley

Zhenhui (Jessie) Li

Chief Scientist, Yunqi Academy of Engineering

Jun Luo

Distinguished Researcher, Huawei Noah’s Ark Lab Canada

Pradeep Varakantham

Associate Professor, Singapore Management University

Hai Yang

Chair Professor, The Hong Kong University of Science and Technology


Liam Paull

University of Montreal & MILA

Zhiwei (Tony) Qin


Rui Song

North Carolina State University, Raleigh

Hongtu Zhu

University of North Carolina, Chapel Hill