# Diagnosing Reinforcement Learning for Traffic Signal Control

arXiv: https://arxiv.org/abs/1905.04716 · HTML: https://mlsanity.com/paper/1905.04716/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2019.

## Built on by (most cited first, 30 of 33)

- [Open-TI: Open Traffic Intelligence with Augmented Language Model](https://mlsanity.com/paper/2401.00211/) (2023)
- [LibSignal: An Open Library for Traffic Signal Control](https://mlsanity.com/paper/2211.10649/) (2022)
- [Multi-Agent Reinforcement Learning for Traffic Signal Control through Universal Communication Method](https://mlsanity.com/paper/2204.12190/) (2022)
- [The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality](https://mlsanity.com/paper/2206.11996/) (2022)
- [MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control](https://mlsanity.com/paper/2101.00746/) (2021)
- [ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/2111.08067/) (2021)
- [Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimization](https://mlsanity.com/paper/2201.00006/) (2021)
- [Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learning](https://mlsanity.com/paper/2303.11899/) (2023)
- [Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem](https://mlsanity.com/paper/2104.10917/) (2021)
- [Learning to Share in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2112.08702/) (2021)
- [Learning to Help Emergency Vehicles Arrive Faster: A Cooperative Vehicle-Road Scheduling Approach](https://mlsanity.com/paper/2202.09773/) (2022)
- [A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Control](https://mlsanity.com/paper/2503.23626/) (2025)
- [GPLight+: A Genetic Programming Method for Learning Symmetric Traffic Signal Control Policy](https://mlsanity.com/paper/2508.16090/) (2025)
- [DataLight: Offline Data-Driven Traffic Signal Control](https://mlsanity.com/paper/2303.10828/) (2023)
- [Generalized Phase Pressure Control Enhanced Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/2503.20205/) (2025)
- [SymLight: Exploring Interpretable and Deployable Symbolic Policies for Traffic Signal Control](https://mlsanity.com/paper/2511.05790/) (2025)
- [CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2603.24366/) (2026)
- [Back to Basics: Deep Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2109.07180/) (2021)
- [DynamicLight: Two-Stage Dynamic Traffic Signal Timing](https://mlsanity.com/paper/2211.01025/) (2022)
- [Real-Time Network-Level Traffic Signal Control: An Explicit Multiagent Coordination Method](https://mlsanity.com/paper/2306.08843/) (2023)
- [A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning](https://mlsanity.com/paper/2311.15920/) (2023)
- [Communication Strategy on Macro-and-Micro Traffic State in Cooperative Deep Reinforcement Learning for Regional Traffic Signal Control](https://mlsanity.com/paper/2502.13248/) (2025)
- [Balancing Multiple Objectives in Urban Traffic Control with Reinforcement Learning from AI Feedback](https://mlsanity.com/paper/2602.20728/) (2026)
- [LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2603.24361/) (2026)
- [A Methodology for the Development of RL-Based Adaptive Traffic Signal Controllers](https://mlsanity.com/paper/2101.09614/) (2021)
- [Learning to Communicate with Reinforcement Learning for an Adaptive Traffic Control System](https://mlsanity.com/paper/2110.15779/) (2021)
- [DynLight: Realize dynamic phase duration with multi-level traffic signal control](https://mlsanity.com/paper/2204.03471/) (2022)
- [Cooperative Reinforcement Learning on Traffic Signal Control](https://mlsanity.com/paper/2205.11291/) (2022)
- [Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction](https://mlsanity.com/paper/2306.09662/) (2023)
- [Purpose in the Machine: Do Traffic Simulators Produce Distributionally Equivalent Outcomes for Reinforcement Learning Applications?](https://mlsanity.com/paper/2311.08429/) (2023)

## Abstract (arXiv)

With the increasing availability of traffic data and advance of deep reinforcement learning techniques, there is an emerging trend of employing reinforcement learning (RL) for traffic signal control. A key question for applying RL to traffic signal control is how to define the reward and state. The ultimate objective in traffic signal control is to minimize the travel time, which is difficult to reach directly. Hence, existing studies often define reward as an ad-hoc weighted linear combination of several traffic measures. However, there is no guarantee that the travel time will be optimized with the reward. In addition, recent RL approaches use more complicated state (e.g., image) in order to describe the full traffic situation. However, none of the existing studies has discussed whether such a complex state representation is necessary. This extra complexity may lead to significantly slower learning process but may not necessarily bring significant performance gain. In this paper, we propose to re-examine the RL approaches through the lens of classic transportation theory. We ask the following questions: (1) How should we design the reward so that one can guarantee to minimize the travel time? (2) How to design a state representation which is concise yet sufficient to obtain the optimal solution? Our proposed method LIT is theoretically supported by the classic traffic signal control methods in transportation field. LIT has a very simple state and reward design, thus can serve as a building block for future RL approaches to traffic signal control. Extensive experiments on both synthetic and real datasets show that our method significantly outperforms the state-of-the-art traffic signal control methods.