# Reinforcement Learning for Traffic Signal Control: Comparison with Commercial Systems

arXiv: https://arxiv.org/abs/2104.10455 · HTML: https://mlsanity.com/paper/2104.10455/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 21 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning](https://mlsanity.com/paper/1704.08883/) (2017)
- [Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network](https://mlsanity.com/paper/1705.02755/) (2017)
- [Intelligent Traffic Light Control Using Distributed Multi-agent Q Learning](https://mlsanity.com/paper/1711.10941/) (2017)
- [Assessment of Reward Functions for Reinforcement Learning Traffic Signal Control under Real-World Limitations](https://mlsanity.com/paper/2008.11634/) (2020)
- [Wavelet Augmented Regression Profiling (WARP): improved long-term estimation of travel time series with recurrent congestion](https://mlsanity.com/paper/2006.13072/) (2020)
- [Assessment of Reward Functions in Reinforcement Learning for Multi-Modal Urban Traffic Control under Real-World limitations](https://mlsanity.com/paper/2010.08819/) (2020)

## Built on by (most cited first, 5 of 5)

- [Learning and Information in Stochastic Networks and Queues](https://mlsanity.com/paper/2105.08769/) (2021)
- [USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agents](https://mlsanity.com/paper/2505.17572/) (2025)
- [Generalisation to unseen topologies: Towards control of biological neural network activity](https://mlsanity.com/paper/2407.12789/) (2024)
- [A Robust and Efficient Multi-Agent Reinforcement Learning Framework for Traffic Signal Control](https://mlsanity.com/paper/2603.12096/) (2026)
- [Preference-Conditioned Multi-Objective Reinforcement Learning for Runtime-Tunable Transit Signal Priority](https://mlsanity.com/paper/2607.18286/) (2026)

## Abstract (arXiv)

Recently, Intelligent Transportation Systems are leveraging the power of increased sensory coverage and computing power to deliver data-intensive solutions achieving higher levels of performance than traditional systems. Within Traffic Signal Control (TSC), this has allowed the emergence of Machine Learning (ML) based systems. Among this group, Reinforcement Learning (RL) approaches have performed particularly well. Given the lack of industry standards in ML for TSC, literature exploring RL often lacks comparison against commercially available systems and straightforward formulations of how the agents operate. Here we attempt to bridge that gap. We propose three different architectures for TSC RL agents and compare them against the currently used commercial systems MOVA, SurTrac and Cyclic controllers and provide pseudo-code for them. The agents use variations of Deep Q-Learning and Actor Critic, using states and rewards based on queue lengths. Their performance is compared in across different map scenarios with variable demand, assessing them in terms of the global delay and average queue length. We find that the RL-based systems can significantly and consistently achieve lower delays when compared with existing commercial systems.