mlsanityData to 7 Oct 2026

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

arXiv 1903.04527 ↗First posted 11 Mar 2019cs.LG stat.MLcited by 159

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. Multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent: advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. Results demonstrate its optimality, robustness, and sample efficiency over other state-of-the-art decentralized MARL algorithms.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 24; 2023: 33; 2024: 17; 2025: 35; 2026: 23.

159 papers.

Sort:

By area:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  3. 2024LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environmentscited by 31
  4. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  5. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  6. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  7. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  8. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  9. 2022Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learningcited by 21
  10. 2021Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learningcited by 20
  11. 2021Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Trafficcited by 20
  12. 2025Mixture-of-Personas Language Models for Population Simulationcited by 20
38 more
  1. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  2. 2022LibSignal: An Open Library for Traffic Signal Controlcited by 16
  3. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 16
  4. 2022Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Trafficcited by 15
  5. 2023Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problemscited by 15
  6. 2021Multi-Agent Imitation Learning with Copulascited by 13
  7. 2021Network-wide traffic signal control optimization using a multi-agent deep reinforcement learningcited by 12
  8. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  9. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12
  10. 2023UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Controlcited by 12
  11. 2021CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehiclescited by 11
  12. 2021DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Controlcited by 11
  13. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 11
  14. 2024X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learnercited by 10
  15. 2025Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learningcited by 10
  16. 2022EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control Systemcited by 9
  17. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  18. 2021A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehiclescited by 8
  19. 2022Consensus Learning for Cooperative Multi-Agent Reinforcement Learningcited by 8
  20. 2022The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Realitycited by 8
  21. 2022Interaction Pattern Disentangling for Multi-Agent Reinforcement Learningcited by 8
  22. 2021A Deep Reinforcement Learning Approach for Fair Traffic Signal Controlcited by 7
  23. 2023DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedbackcited by 7
  24. 2023Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Controlcited by 7
  25. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  26. 2021MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Controlcited by 6
  27. 2021Towards Real-World Deployment of Reinforcement Learning for Traffic Signal Controlcited by 6
  28. 2022Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Gamescited by 6
  29. 2023Efficient Planning in Combinatorial Action Spaces with Applications to Cooperative Multi-Agent Reinforcement Learningcited by 6
  30. 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
  31. 2023Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specificationscited by 6
  32. 2023Adaptive parameter sharing for multi-agent reinforcement learningcited by 6
  33. 2021Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problemcited by 5
  34. 2021Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-levelcited by 5
  35. 2022CoTV: Cooperative Control for Traffic Light Signals and Connected Autonomous Vehicles using Deep Reinforcement Learningcited by 5
  36. 2022Communication-Efficient Actor-Critic Methods for Homogeneous Markov Gamescited by 5
  37. 2022Model-based graph reinforcement learning for inductive traffic signal controlcited by 5
  38. 2022Reinforcement learning for traffic signal control in hybrid action spacecited by 5

50 most cited of 159 All 159 (JSON)