Communication Strategy on Macro-and-Micro Traffic State in Cooperative Deep Reinforcement Learning for Regional Traffic Signal Control
Not on the map. Why?
Abstract (arXiv)
Adaptive Traffic Signal Control (ATSC) has become a popular research topic in intelligent transportation systems. Regional Traffic Signal Control (RTSC) using the Multi-agent Deep Reinforcement Learning (MADRL) technique has become a promising approach for ATSC due to its ability to achieve the optimum trade-off between scalability and optimality. Most existing RTSC approaches partition a traffic network into several disjoint regions, followed by applying centralized reinforcement learning techniques to each region. However, the pursuit of cooperation among RTSC agents still remains an open issue and no communication strategy for RTSC agents has been investigated. In this paper, we propose communication strategies to capture the correlation of micro-traffic states among lanes and the correlation of macro-traffic states among intersections. We first justify the evolution equation of the RTSC process is Markovian via a system of store-and-forward queues. Next, based on the evolution equation, we propose two GAT-Aggregated (GA2) communication modules--GA2-Naive and GA2-Aug to extract both intra-region and inter-region correlations between macro and micro traffic states. While GA2-Naive only considers the movements at each intersection, GA2-Aug also considers the lane-changing behavior of vehicles. Two proposed communication modules are then aggregated into two existing novel RTSC frameworks--RegionLight and Regional-DRL. Experimental results demonstrate that both GA2-Naive and GA2-Aug effectively improve the performance of existing RTSC frameworks under both real and synthetic scenarios. Hyperparameter testing also reveals the robustness and potential of our communication modules in large-scale traffic networks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 53 references that are in the map. First posted on arXiv 18 Feb 2025.
Builds on
20 of 53 references.
References · 20
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
- 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 158
- 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
- 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
- 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 133
- 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 114
- 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
- 2020Deep Reinforcement Learning for Intelligent Transportation Systems: A Surveycited by 94
- 2019Deep Reinforcement Learning for Cyber Securitycited by 92
- 2017Action Branching Architectures for Deep Reinforcement Learningcited by 90
8 more
- 2019A Survey on Traffic Signal Control Methodscited by 78
- 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 51
- 2017Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learningcited by 33
- 2019Diagnosing Reinforcement Learning for Traffic Signal Controlcited by 33
- 2016Using a Deep Reinforcement Learning Agent for Traffic Signal Controlcited by 26
- 2021Efficient Pressure: Improving efficiency for signalized intersectionscited by 24
- 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
- 2023Large-Scale Traffic Signal Control by a Nash Deep Q-network Approachcited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.