mlsanityData to 7 Oct 2026

Offline Reinforcement Learning for Road Traffic Control

arXiv 2201.02381 ↗First posted 7 Jan 2022cs.AI cs.LGcited by 5

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Traffic signal control is an important problem in urban mobility with a significant potential of economic and environmental impact. While there is a growing interest in Reinforcement Learning (RL) for traffic signal control, the work so far has focussed on learning through simulations which could lead to inaccuracies due to simplifying assumptions. Instead, real experience data on traffic is available and could be exploited at minimal costs. Recent progress in offline or batch RL has enabled just that. Model-based offline RL methods, in particular, have been shown to generalize from the experience data much better than others. We build a model-based learning framework which infers a Markov Decision Process (MDP) from a dataset collected using a cyclic traffic signal control policy that is both commonplace and easy to gather. The MDP is built with pessimistic costs to manage out-of-distribution scenarios using an adaptive shaping of rewards which is shown to provide better regularization compared to the prior related work in addition to being PAC-optimal. Our model is evaluated on a complex signalized roundabout showing that it is possible to build highly performant traffic control policies in a data efficient manner.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 28 of its 59 references that are in the map. First posted on arXiv 7 Jan 2022.

Builds on

28 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement13 Oct

Other references · 27

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  9. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  10. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  11. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
15 more
  1. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  3. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  4. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  5. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  6. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
  7. 2020Model-Based Offline Planningcited by 126
  8. 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 114
  9. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  10. 2019A Survey on Traffic Signal Control Methodscited by 78
  11. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 72
  12. 2017Flow: A Modular Learning Framework for Mixed Autonomy Trafficcited by 67
  13. 2017Dissipation of stop-and-go waves via control of autonomous vehicles: Field experimentscited by 41
  14. 2020DeepAveragers: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPscited by 16
  15. 2021Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Criticcited by 11

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 2.

5 papers.

  1. 2024SynTraC: A Synthetic Dataset for Traffic Signal Control from Traffic Monitoring Camerascited by 4
  2. 2023Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approachcited by 3
  3. 2023Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learningcited by 2
  4. 2024Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharingcited by 2
  5. 2023A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024OffLight: An Offline Multi-Agent Reinforcement Learning Framework for Traffic Signal Control
  2. 2025Robust Single-Agent Reinforcement Learning for Regional Traffic Signal Control Under Demand Fluctuations
  3. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
  4. 2022Estimating Link Flows in Road Networks with Synthetic Trajectory Data Generation: Reinforcement Learning-based Approaches
  5. 2025Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems
  6. 2024Queue-based Eco-Driving at Roundabouts with Reinforcement Learning
  7. 2022Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning
  8. 2025Evolutionary Discovery of Heuristic Policies for Traffic Signal Control