# Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

arXiv: https://arxiv.org/abs/2104.09936 · HTML: https://mlsanity.com/paper/2104.09936/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [R-MADDPG for Partially Observable Environments and Limited Communication](https://mlsanity.com/paper/2002.06684/) (2020)
- [Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning](https://mlsanity.com/paper/1704.08883/) (2017)
- [Deep Deterministic Policy Gradient for Urban Traffic Light Control](https://mlsanity.com/paper/1703.09035/) (2017)
- [Using a Deep Reinforcement Learning Agent for Traffic Signal Control](https://mlsanity.com/paper/1611.01142/) (2016)
- [Using Reinforcement Learning with Partial Vehicle Detection for Intelligent Traffic Signal Control](https://mlsanity.com/paper/1807.01628/) (2018)
- [Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov Decision Process and Imitation Learning](https://mlsanity.com/paper/1905.09906/) (2019)
- [Partially Detected Intelligent Traffic Signal Control: Environmental Adaptation](https://mlsanity.com/paper/1910.10808/) (2019)

## Built on by (most cited first, 12 of 12)

- [Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions](https://mlsanity.com/paper/2501.04437/) (2025)
- [The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality](https://mlsanity.com/paper/2206.11996/) (2022)
- [Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?](https://mlsanity.com/paper/2407.07364/) (2024)
- [Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies](https://mlsanity.com/paper/2410.10212/) (2024)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Self-Regulating Cars: Automating Traffic Control in Free Flow Road Networks](https://mlsanity.com/paper/2506.11973/) (2025)
- [Human-Centric Traffic Signal Control for Equity: A Multi-Agent Action Branching Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2602.02959/) (2026)
- [Dynamic network congestion pricing based on deep reinforcement learning](https://mlsanity.com/paper/2206.12188/) (2022)
- [Exploring the impact of traffic signal control and connected and automated vehicles on intersections safety: A deep reinforcement learning approach](https://mlsanity.com/paper/2405.19236/) (2024)
- [Smart Traffic Signals: Comparing MARL and Fixed-Time Strategies](https://mlsanity.com/paper/2505.14544/) (2025)
- [A Hierarchical Signal Coordination and Control System Using a Hybrid Model-based and Reinforcement Learning Approach](https://mlsanity.com/paper/2508.20102/) (2025)
- [Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal control](https://mlsanity.com/paper/2602.17068/) (2026)

## Abstract (arXiv)

Inefficient traffic control may cause numerous problems such as traffic congestion and energy waste. This paper proposes a novel multi-agent reinforcement learning method, named KS-DDPG (Knowledge Sharing Deep Deterministic Policy Gradient) to achieve optimal control by enhancing the cooperation between traffic signals. By introducing the knowledge-sharing enabled communication protocol, each agent can access to the collective representation of the traffic environment collected by all agents. The proposed method is evaluated through two experiments respectively using synthetic and real-world datasets. The comparison with state-of-the-art reinforcement learning-based and conventional transportation methods demonstrate the proposed KS-DDPG has significant efficiency in controlling large-scale transportation networks and coping with fluctuations in traffic flow. In addition, the introduced communication mechanism has also been proven to speed up the convergence of the model without significantly increasing the computational burden.