mlsanityData to 7 Oct 2026

Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning

arXiv 1908.03761 ↗First posted 10 Aug 2019cs.LG cs.MA stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Finding the optimal signal timing strategy is a difficult task for the problem of large-scale traffic signal control (TSC). Multi-Agent Reinforcement Learning (MARL) is a promising method to solve this problem. However, there is still room for improvement in extending to large-scale problems and modeling the behaviors of other agents for each individual agent. In this paper, a new MARL, called Cooperative double Q-learning (Co-DQL), is proposed, which has several prominent features. It uses a highly scalable independent double Q-learning method based on double estimators and the UCB policy, which can eliminate the over-estimation problem existing in traditional independent Q-learning while ensuring exploration. It uses mean field approximation to model the interaction among agents, thereby making agents learn a better cooperative strategy. In order to improve the stability and robustness of the learning process, we introduce a new reward allocation mechanism and a local state sharing method. In addition, we analyze the convergence properties of the proposed algorithm. Co-DQL is applied on TSC and tested on a multi-traffic signal simulator. According to the results obtained on several traffic scenarios, Co- DQL outperforms several state-of-the-art decentralized MARL algorithms. It can effectively shorten the average waiting time of the vehicles in the whole road system.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 4; 2023: 9; 2024: 4; 2025: 5; 2026: 5.

33 papers.

Sort:

By area:

  1. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  2. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 31
  3. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Surveycited by 14
  4. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  5. 2021ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Controlcited by 8
  6. 2023Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global Statecited by 8
  7. 2023DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedbackcited by 7
  8. 2023Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Controlcited by 7
  9. 2022Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?cited by 6
  10. 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
  11. 2022Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)cited by 5
  12. 2024Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphscited by 5
21 more
  1. 2021Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baselinecited by 4
  2. 2022On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learningcited by 4
  3. 2023Large-Scale Traffic Signal Control by a Nash Deep Q-network Approachcited by 4
  4. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4
  5. 2023Comparing Reinforcement Learning and Human Learning using the Game of Hidden Rulescited by 2
  6. 2024ROMA-iQSS: An Objective Alignment Approach via State-Based Value Learning and ROund-Robin Multi-Agent Schedulingcited by 1
  7. 2024Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learningcited by 1
  8. 2025Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Controlcited by 1
  9. 2026Human-Centric Traffic Signal Control for Equity: A Multi-Agent Action Branching Deep Reinforcement Learning Approachcited by 1
  10. 2021Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learningcited by 0
  11. 2021A Reinforcement Learning-based Adaptive Control Model for Future Street Planning, An Algorithm and A Case Studycited by 0
  12. 2021CGIBNet: Bandwidth-constrained Communication with Graph Information Bottleneck in Multi-Agent Reinforcement Learningcited by 0
  13. 2022A Decentralized Communication Framework based on Dual-Level Recurrence for Multi-Agent Reinforcement Learningcited by 0
  14. 2023Traffic Shaping and Hysteresis Mitigation Using Deep Reinforcement Learning in a Connected Driving Environmentcited by 0
  15. 2023Heterogeneous Multi-Agent Reinforcement Learning via Mirror Descent Policy Optimizationcited by 0
  16. 2025Toward Dependency Dynamics in Multi-Agent Reinforcement Learning for Traffic Signal Controlcited by 0
  17. 2025Fully Decentralized Cooperative Multi-Agent Reinforcement Learning is A Context Modeling Problemcited by 0
  18. 2026Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal controlcited by 0
  19. 2026Distributed Real-Time Vehicle Control for Emergency Vehicle Transit: A Scalable Cooperative Methodcited by 0
  20. 2026Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedbackcited by 0
  21. 2026SIGMA: Symmetry-aware, Intelligent, Geometric, Multi-objective Adaptive Control for Robust, Dependable Traffic Managementcited by 0