mlsanityData to 14 Oct 2026

Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control

arXiv 2310.02435 ↗First posted 3 Oct 2023cs.MA cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Traffic signal control (TSC) is a challenging problem within intelligent transportation systems and has been tackled using multi-agent reinforcement learning (MARL). While centralized approaches are often infeasible for large-scale TSC problems, decentralized approaches provide scalability but introduce new challenges, such as partial observability. Communication plays a critical role in decentralized MARL, as agents must learn to exchange information using messages to better understand the system and achieve effective coordination. Deep MARL has been used to enable inter-agent communication by learning communication protocols in a differentiable manner. However, many deep MARL communication frameworks proposed for TSC allow agents to communicate with all other agents at all times, which can add to the existing noise in the system and degrade overall performance. In this study, we propose a communication-based MARL framework for large-scale TSC. Our framework allows each agent to learn a communication policy that dictates "which" part of the message is sent "to whom". In essence, our framework enables agents to selectively choose the recipients of their messages and exchange variable length messages with them. This results in a decentralized and flexible communication mechanism in which agents can effectively use the communication channel only when necessary. We designed two networks, a synthetic 4 × 4 grid network and a real-world network based on the Pasubio neighborhood in Bologna. Our framework achieved the lowest network congestion compared to related methods, with agents utilizing ~ 47-65 % of the communication channel. Ablation studies further demonstrated the effectiveness of the communication policies learned within our framework.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 34 of its 80 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

34 of 80 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  6. 2016Deep Variational Information Bottleneckcited by 930
  7. 2019The StarCraft Multi-Agent Challengecited by 597
  8. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  9. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  10. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  11. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  12. 2016Learning Multiagent Communication with Backpropagationcited by 366
22 more
  1. 2019Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcited by 164
  2. 2018Graph Convolutional Reinforcement Learningcited by 152
  3. 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 138
  4. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 121
  5. 2020Deep Reinforcement Learning for Intelligent Transportation Systems: A Surveycited by 98
  6. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 93
  7. 2019Learning Phase Competition for Traffic Signal Controlcited by 87
  8. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 86
  9. 2018Learning Disentangled Joint Continuous and Discrete Representationscited by 83
  10. 2019Multi-Agent Game Abstraction via Graph Attention Neural Networkcited by 82
  11. 2019A Survey on Traffic Signal Control Methodscited by 79
  12. 2019On the Pitfalls of Measuring Emergent Communicationcited by 71
  13. 2017VAIN: Attentional Multi-agent Predictive Modelingcited by 53
  14. 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 53
  15. 2019Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Controlcited by 46
  16. 2019Learning Transferable Cooperative Behavior in Multi-Agent Teamscited by 44
  17. 2019Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learningcited by 33
  18. 2016Using a Deep Reinforcement Learning Agent for Traffic Signal Controlcited by 26
  19. 2020IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Controlcited by 22
  20. 2019Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communicationcited by 18
  21. 2020Networked Multi-Agent Reinforcement Learning with Emergent Communicationcited by 13
  22. 2018An Efficient Deep Reinforcement Learning Model for Urban Traffic Controlcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 3.

7 papers.

  1. 2024OffLight: An Offline Multi-Agent Reinforcement Learning Framework for Traffic Signal Controlcited by 3
  2. 2024Integrating Transit Signal Priority into Multi-Agent Reinforcement Learning based Traffic Signal Controlcited by 2
  3. 2026Human-Centric Traffic Signal Control for Equity: A Multi-Agent Action Branching Deep Reinforcement Learning Approachcited by 1
  4. 2025Robust Single-Agent Reinforcement Learning for Regional Traffic Signal Control Under Demand Fluctuationscited by 0
  5. 2025Single-agent Reinforcement Learning Model for Regional Adaptive Traffic Signal Controlcited by 0
  6. 2026Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal controlcited by 0
  7. 2026Explainable Reinforcement Learning for Adaptive Traffic Signal Controlcited by 0