# Deep Deterministic Policy Gradient for Urban Traffic Light Control

arXiv: https://arxiv.org/abs/1703.09035 · HTML: https://mlsanity.com/paper/1703.09035/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2017.

## Built on by (most cited first, 30 of 30)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic](https://mlsanity.com/paper/2306.02865/) (2023)
- [UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control](https://mlsanity.com/paper/2312.05090/) (2023)
- [Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning](https://mlsanity.com/paper/2104.09936/) (2021)
- [Multi-hop RIS-Empowered Terahertz Communications: A DRL-based Hybrid Beamforming Design](https://mlsanity.com/paper/2101.09137/) (2021)
- [Evolutionary Action Selection for Gradient-based Policy Learning](https://mlsanity.com/paper/2201.04286/) (2022)
- [SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2305.16145/) (2023)
- [Model-based graph reinforcement learning for inductive traffic signal control](https://mlsanity.com/paper/2208.00659/) (2022)
- [Reinforcement learning for traffic signal control in hybrid action space](https://mlsanity.com/paper/2211.12956/) (2022)
- [Boosted Genetic Algorithm using Machine Learning for traffic control optimization](https://mlsanity.com/paper/2103.08317/) (2021)
- [Hierarchical Robot Navigation in Novel Environments using Rough 2-D Maps](https://mlsanity.com/paper/2106.03665/) (2021)
- [Large-Scale Traffic Signal Control by a Nash Deep Q-network Approach](https://mlsanity.com/paper/2301.00637/) (2023)
- [Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2108.04087/) (2021)
- [HumanLight: Incentivizing Ridesharing via Human-centric Deep Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2304.03697/) (2023)
- [A Simple Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation](https://mlsanity.com/paper/2403.06884/) (2024)
- [A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization](https://mlsanity.com/paper/2107.06115/) (2021)
- [Attacking Deep Reinforcement Learning-Based Traffic Signal Control Systems with Colluding Vehicles](https://mlsanity.com/paper/2111.02845/) (2021)
- [GrASP: Gradient-Based Affordance Selection for Planning](https://mlsanity.com/paper/2202.04772/) (2022)
- [Adaptive Transit Signal Priority based on Deep Reinforcement Learning and Connected Vehicles in a Traffic Microsimulation Environment](https://mlsanity.com/paper/2408.00098/) (2024)
- [Integrating Transit Signal Priority into Multi-Agent Reinforcement Learning based Traffic Signal Control](https://mlsanity.com/paper/2411.19359/) (2024)
- [CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2603.24366/) (2026)
- [Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks](https://mlsanity.com/paper/2212.05572/) (2022)
- [Adaptive Road Configurations for Improved Autonomous Vehicle-Pedestrian Interactions using Reinforcement Learning](https://mlsanity.com/paper/2303.12289/) (2023)
- [Truncating Trajectories in Monte Carlo Reinforcement Learning](https://mlsanity.com/paper/2305.04361/) (2023)
- [CycLight: learning traffic signal cooperation with a cycle-level strategy](https://mlsanity.com/paper/2401.08121/) (2024)
- [GuideLight: "Industrial Solution" Guidance for More Practical Traffic Signal Control Agents](https://mlsanity.com/paper/2407.10811/) (2024)
- [Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction](https://mlsanity.com/paper/2306.09662/) (2023)
- [A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Control](https://mlsanity.com/paper/2503.14250/) (2025)
- [A Semi Centralized Training Decentralized Execution Architecture for Multi Agent Deep Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2512.04653/) (2025)
- [Explainable Reinforcement Learning for Adaptive Traffic Signal Control](https://mlsanity.com/paper/2607.03703/) (2026)

## Abstract (arXiv)

Traffic light timing optimization is still an active line of research despite the wealth of scientific literature on the topic, and the problem remains unsolved for any non-toy scenario. One of the key issues with traffic light optimization is the large scale of the input information that is available for the controlling agent, namely all the traffic data that is continually sampled by the traffic detectors that cover the urban network. This issue has in the past forced researchers to focus on agents that work on localized parts of the traffic network, typically on individual intersections, and to coordinate every individual agent in a multi-agent setup. In order to overcome the large scale of the available state information, we propose to rely on the ability of deep Learning approaches to handle large input spaces, in the form of Deep Deterministic Policy Gradient (DDPG) algorithm. We performed several experiments with a range of models, from the very simple one (one intersection) to the more complex one (a big city section).