# Improving the generalizability and robustness of large-scale traffic signal control

arXiv: https://arxiv.org/abs/2306.01925 · HTML: https://mlsanity.com/paper/2306.01925/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 55 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Noisy Networks for Exploration](https://mlsanity.com/paper/1706.10295/) (2017)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [Inductive Graph Neural Networks for Spatiotemporal Kriging](https://mlsanity.com/paper/2006.07527/) (2020)
- [Multi-Agent Game Abstraction via Graph Attention Neural Network](https://mlsanity.com/paper/1911.10715/) (2019)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [A Comparative Analysis of Expected and Distributional Reinforcement Learning](https://mlsanity.com/paper/1901.11084/) (2019)
- [Low-Rank Autoregressive Tensor Completion for Spatiotemporal Traffic Data Imputation](https://mlsanity.com/paper/2104.14936/) (2021)
- [IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control](https://mlsanity.com/paper/2003.05738/) (2020)
- [GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning](https://mlsanity.com/paper/2009.08052/) (2020)
- [Driving Decision and Control for Autonomous Lane Change based on Deep Reinforcement Learning](https://mlsanity.com/paper/1904.10171/) (2019)
- [Using Reinforcement Learning with Partial Vehicle Detection for Intelligent Traffic Signal Control](https://mlsanity.com/paper/1807.01628/) (2018)
- [MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control](https://mlsanity.com/paper/2101.00746/) (2021)
- [Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures](https://mlsanity.com/paper/1904.08353/) (2019)
- [Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline](https://mlsanity.com/paper/2101.09640/) (2021)

## Built on by (most cited first, 5 of 5)

- [Multi-hop Upstream Anticipatory Traffic Signal Control with Deep Reinforcement Learning](https://mlsanity.com/paper/2411.07271/) (2024)
- [Graph Neural Networks in Intelligent Transportation Systems: Advances, Applications and Trends](https://mlsanity.com/paper/2401.00713/) (2024)
- [Robustness of Reinforcement Learning-Based Traffic Signal Control under Incidents: A Comparative Study](https://mlsanity.com/paper/2506.13836/) (2025)
- [Reinforcement Learning-Based Traffic Signal Control for IoT-Enabled Intersections](https://mlsanity.com/paper/2606.22108/) (2026)
- [Explainable Reinforcement Learning for Adaptive Traffic Signal Control](https://mlsanity.com/paper/2607.03703/) (2026)

## Abstract (arXiv)

A number of deep reinforcement-learning (RL) approaches propose to control traffic signals. In this work, we study the robustness of such methods along two axes. First, sensor failures and GPS occlusions create missing-data challenges and we show that recent methods remain brittle in the face of these missing data. Second, we provide a more systematic study of the generalization ability of RL methods to new networks with different traffic regimes. Again, we identify the limitations of recent approaches. We then propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble. Building upon the state-of-the-art previous model which uses a decentralized approach for large-scale traffic signal control with graph convolutional networks (GCNs), we first learn models using a distributional reinforcement learning (DisRL) approach. In particular, we use implicit quantile networks (IQN) to model the state-action return distribution with quantile regression. For traffic signal control problems, an ensemble of standard RL and DisRL yields superior performance across different scenarios, including different levels of missing sensor data and traffic flow patterns. Furthermore, the learning scheme of the resulting model can improve zero-shot transferability to different road network structures, including both synthetic networks and real-world networks (e.g., Luxembourg, Manhattan). We conduct extensive experiments to compare our approach to multi-agent reinforcement learning and traditional transportation approaches. Results show that the proposed method improves robustness and generalizability in the face of missing data, varying road networks, and traffic flows.