# Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

arXiv: https://arxiv.org/abs/1704.08883 · HTML: https://mlsanity.com/paper/1704.08883/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2017.

## Built on by (most cited first, 30 of 33)

- [Mixture-of-Personas Language Models for Population Simulation](https://mlsanity.com/paper/2504.05019/) (2025)
- [Provably Efficient Policy Optimization for Two-Player Zero-Sum Markov Games](https://mlsanity.com/paper/2102.08903/) (2021)
- [UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control](https://mlsanity.com/paper/2312.05090/) (2023)
- [Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning](https://mlsanity.com/paper/2104.09936/) (2021)
- [The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality](https://mlsanity.com/paper/2206.11996/) (2022)
- [SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2305.16145/) (2023)
- [Reinforcement Learning for Traffic Signal Control: Comparison with Commercial Systems](https://mlsanity.com/paper/2104.10455/) (2021)
- [Reinforcement learning for traffic signal control in hybrid action space](https://mlsanity.com/paper/2211.12956/) (2022)
- [Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline](https://mlsanity.com/paper/2101.09640/) (2021)
- [Design and implementation of an environment for Learning to Run a Power Network (L2RPN)](https://mlsanity.com/paper/2104.04080/) (2021)
- [ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning](https://mlsanity.com/paper/2210.13378/) (2022)
- [Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems](https://mlsanity.com/paper/2302.03669/) (2023)
- [Knowing What Not to Do: Leverage Language Model Insights for Action Space Pruning in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2405.16854/) (2024)
- [A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Control](https://mlsanity.com/paper/2503.23626/) (2025)
- [A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization](https://mlsanity.com/paper/2107.06115/) (2021)
- [Attacking Deep Reinforcement Learning-Based Traffic Signal Control Systems with Colluding Vehicles](https://mlsanity.com/paper/2111.02845/) (2021)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2603.24366/) (2026)
- [Real-Time Network-Level Traffic Signal Control: An Explicit Multiagent Coordination Method](https://mlsanity.com/paper/2306.08843/) (2023)
- [GuideLight: "Industrial Solution" Guidance for More Practical Traffic Signal Control Agents](https://mlsanity.com/paper/2407.10811/) (2024)
- [Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion](https://mlsanity.com/paper/2408.15751/) (2024)
- [Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learning](https://mlsanity.com/paper/2412.03925/) (2024)
- [Communication Strategy on Macro-and-Micro Traffic State in Cooperative Deep Reinforcement Learning for Regional Traffic Signal Control](https://mlsanity.com/paper/2502.13248/) (2025)
- [Deep Reinforcement Learning-Based Product Recommender for Online Advertising](https://mlsanity.com/paper/2102.00333/) (2021)
- [Cooperative Reinforcement Learning on Traffic Signal Control](https://mlsanity.com/paper/2205.11291/) (2022)
- [Traffic Management of Autonomous Vehicles using Policy Based Deep Reinforcement Learning and Intelligent Routing](https://mlsanity.com/paper/2206.14608/) (2022)
- [Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction](https://mlsanity.com/paper/2306.09662/) (2023)
- [When Mining Electric Locomotives Meet Reinforcement Learning](https://mlsanity.com/paper/2311.08153/) (2023)
- [A DRL solution to help reduce the cost in waiting time of securing a traffic light for cyclists](https://mlsanity.com/paper/2311.13905/) (2023)
- [An Offline Meta Black-box Optimization Framework for Adaptive Design of Urban Traffic Light Management Systems](https://mlsanity.com/paper/2408.07327/) (2024)

## Abstract (arXiv)

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process.