# Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

arXiv: https://arxiv.org/abs/2311.03756 · HTML: https://mlsanity.com/paper/2311.03756/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 13 of 13)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017)
- [Convolutional Neural Network Architectures for Signals Supported on Graphs](https://mlsanity.com/paper/1805.00165/) (2018)
- [Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks](https://mlsanity.com/paper/1903.10527/) (2019)
- [Learning Optimal Resource Allocations in Wireless Systems](https://mlsanity.com/paper/1807.08088/) (2018)
- [Discrete Signal Processing on Graphs: Sampling Theory](https://mlsanity.com/paper/1503.05432/) (2015)
- [Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence](https://mlsanity.com/paper/2103.10290/) (2021)
- [DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Control](https://mlsanity.com/paper/2109.05491/) (2021)
- [DRLE: Decentralized Reinforcement Learning at the Edge for Traffic Light Control in the IoV](https://mlsanity.com/paper/2009.01502/) (2020)
- [A pedestrian hopping model and traffic light scheduling for pedestrian-vehicle mixed-flow networks](https://mlsanity.com/paper/1705.05251/) (2017)

## Built on by (most cited first, 2 of 2)

- [LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systems](https://mlsanity.com/paper/2511.19368/) (2025)
- [Decentralized Multi-agent Reinforcement Learning for Resilient Critical Infrastructures](https://mlsanity.com/paper/2607.18359/) (2026)

## Abstract (arXiv)

This paper considers optimal traffic signal control in smart cities, which has been taken as a complex networked system control problem. Given the interacting dynamics among traffic lights and road networks, attaining controller adaptivity and scalability stands out as a primary challenge. Capturing the spatial-temporal correlation among traffic lights under the framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution. Nevertheless, existing MARL algorithms ignore effective information aggregation which is fundamental for improving the learning capacity of decentralized agents. In this paper, we design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop a topology-aware information aggregation strategy to extract correlation-related information from unstructured data gathered in the road network. Particularly, we transfer the road network topology into a graph shift operator by forming a diffusion process on the topology, which subsequently facilitates the construction of graph signals. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning. Extensive experiments based on both synthetic and real-world datasets verify that our proposal outperforms existing decentralized algorithms.