# The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

arXiv: https://arxiv.org/abs/2206.11996 · HTML: https://mlsanity.com/paper/2206.11996/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 140 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Reinforcement Learning for Solving the Vehicle Routing Problem](https://mlsanity.com/paper/1802.04240/) (2018)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning](https://mlsanity.com/paper/2007.04938/) (2020)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey](https://mlsanity.com/paper/2005.00935/) (2020)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [A Survey on Traffic Signal Control Methods](https://mlsanity.com/paper/1904.08117/) (2019)
- [Constrained Variational Policy Optimization for Safe Reinforcement Learning](https://mlsanity.com/paper/2201.11927/) (2022)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks](https://mlsanity.com/paper/1803.11115/) (2018)
- [Value constrained model-free continuous control](https://mlsanity.com/paper/1902.04623/) (2019)
- [Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning](https://mlsanity.com/paper/1704.08883/) (2017)
- [Diagnosing Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/1905.04716/) (2019)
- [Using a Deep Reinforcement Learning Agent for Traffic Signal Control](https://mlsanity.com/paper/1611.01142/) (2016)
- [Learning an Interpretable Traffic Signal Control Policy](https://mlsanity.com/paper/1912.11023/) (2019)
- [Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning](https://mlsanity.com/paper/2104.09936/) (2021)
- [CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehicles](https://mlsanity.com/paper/2104.10340/) (2021)
- [Multi-Agent Reinforcement Learning for Traffic Signal Control through Universal Communication Method](https://mlsanity.com/paper/2204.12190/) (2022)
- [A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2111.00278/) (2021)
- [Intelligent Traffic Light Control Using Distributed Multi-agent Q Learning](https://mlsanity.com/paper/1711.10941/) (2017)
- [Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures](https://mlsanity.com/paper/1904.08353/) (2019)

## Built on by (most cited first, 9 of 9)

- [Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems](https://mlsanity.com/paper/2508.02344/) (2025)
- [OffLight: An Offline Multi-Agent Reinforcement Learning Framework for Traffic Signal Control](https://mlsanity.com/paper/2411.06601/) (2024)
- [Bridging the Reality Gap of Reinforcement Learning based Traffic Signal Control using Domain Randomization and Meta Learning](https://mlsanity.com/paper/2307.11357/) (2023)
- [PyTSC: A Unified Platform for Multi-Agent Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2410.18202/) (2024)
- [An Offline Meta Black-box Optimization Framework for Adaptive Design of Urban Traffic Light Management Systems](https://mlsanity.com/paper/2408.07327/) (2024)
- [Robustness of Reinforcement Learning-Based Traffic Signal Control under Incidents: A Comparative Study](https://mlsanity.com/paper/2506.13836/) (2025)
- [Active Inference for Adaptive Traffic Signal Control in Noisy Nonstationary IoT Environments](https://mlsanity.com/paper/2606.13698/) (2026)
- [Sim2Signal: Sim-to-Real Benchmarks for Traffic Signal Control](https://mlsanity.com/paper/2609.01676/) (2026)
- [EvoSignal: LLM-Guided Evolutionary Design of Modular Traffic Signal Control Programs](https://mlsanity.com/paper/2610.09563/) (2026)

## Abstract (arXiv)

Traffic signal control (TSC) is a high-stakes domain that is growing in importance as traffic volume grows globally. An increasing number of works are applying reinforcement learning (RL) to TSC; RL can draw on an abundance of traffic data to improve signalling efficiency. However, RL-based signal controllers have never been deployed. In this work, we provide the first review of challenges that must be addressed before RL can be deployed for TSC. We focus on four challenges involving (1) uncertainty in detection, (2) reliability of communications, (3) compliance and interpretability, and (4) heterogeneous road users. We show that the literature on RL-based TSC has made some progress towards addressing each challenge. However, more work should take a systems thinking approach that considers the impacts of other pipeline components on RL.