# Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

arXiv: https://arxiv.org/abs/2304.10722 · HTML: https://mlsanity.com/paper/2304.10722/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Information-Theoretic Considerations in Batch Reinforcement Learning](https://mlsanity.com/paper/1905.00360/) (2019)
- [Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction](https://mlsanity.com/paper/1802.08714/) (2018)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Forecasting Network-wide Traffic State with Missing Values](https://mlsanity.com/paper/2005.11627/) (2020)
- [DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Control](https://mlsanity.com/paper/2109.05491/) (2021)
- [ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/2111.08067/) (2021)
- [Resource Management for Transmit Power Minimization in UAV-Assisted RIS HetNets Supported by Dual Connectivity](https://mlsanity.com/paper/2106.13174/) (2021)
- [Inverse Reinforcement Learning with Missing Data](https://mlsanity.com/paper/1911.06930/) (2019)

## Built on by (most cited first, 13 of 13)

- [CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic](https://mlsanity.com/paper/2410.14368/) (2024)
- [MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctions](https://mlsanity.com/paper/2408.09768/) (2024)
- [DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data](https://mlsanity.com/paper/2410.22938/) (2024)
- [VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture](https://mlsanity.com/paper/2505.19486/) (2025)
- [Shaded Route Planning Using Active Segmentation and Identification of Satellite Images](https://mlsanity.com/paper/2407.13689/) (2024)
- [FuzzyLight: A Robust Two-Stage Fuzzy Approach for Traffic Signal Control Works in Real Cities](https://mlsanity.com/paper/2501.15820/) (2025)
- [Planning Under Observation Mismatch for Traffic Signal Control via Adaptive Modular World Models](https://mlsanity.com/paper/2501.02548/) (2025)
- [Harnessing Generative LLMs for Enhanced Financial Event Entity Extraction Performance](https://mlsanity.com/paper/2504.14633/) (2025)
- [HALO: Hierarchical Reinforcement Learning for Large-Scale Adaptive Traffic Signal Control](https://mlsanity.com/paper/2506.14391/) (2025)
- [An Interdisciplinary and Cross-Task Review on Missing Data Imputation](https://mlsanity.com/paper/2511.01196/) (2025)
- [PAST: A Primary-Auxiliary Spatio-Temporal Network for Traffic Time Series Imputation](https://mlsanity.com/paper/2511.13414/) (2025)
- [Admissable: Training Reinforcement Learning Agents against Adversarial Missingness](https://mlsanity.com/paper/2609.15297/) (2026)
- [VLALight: A Vision-Language-Action Model for Traffic Signal Control](https://mlsanity.com/paper/2609.36934/) (2026)

## Abstract (arXiv)

The emergence of reinforcement learning (RL) methods in traffic signal control tasks has achieved better performance than conventional rule-based approaches. Most RL approaches require the observation of the environment for the agent to decide which action is optimal for a long-term reward. However, in real-world urban scenarios, missing observation of traffic states may frequently occur due to the lack of sensors, which makes existing RL methods inapplicable on road networks with missing observation. In this work, we aim to control the traffic signals in a real-world setting, where some of the intersections in the road network are not installed with sensors and thus with no direct observations around them. To the best of our knowledge, we are the first to use RL methods to tackle the traffic signal control problem in this real-world setting. Specifically, we propose two solutions: the first one imputes the traffic states to enable adaptive control, and the second one imputes both states and rewards to enable adaptive control and the training of RL agents. Through extensive experiments on both synthetic and real-world road network traffic, we reveal that our method outperforms conventional approaches and performs consistently with different missing rates. We also provide further investigations on how missing data influences the performance of our model.