# Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimization

arXiv: https://arxiv.org/abs/2201.00006 · HTML: https://mlsanity.com/paper/2201.00006/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 30 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [Diagnosing Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/1905.04716/) (2019)
- [Decentralized Signal Control for Urban Road Networks](https://mlsanity.com/paper/1310.0491/) (2013)

## Built on by (most cited first, 7 of 7)

- [SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2305.16145/) (2023)
- [DataLight: Offline Data-Driven Traffic Signal Control](https://mlsanity.com/paper/2303.10828/) (2023)
- [Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2503.11488/) (2025)
- [CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2603.24366/) (2026)
- [DynamicLight: Two-Stage Dynamic Traffic Signal Timing](https://mlsanity.com/paper/2211.01025/) (2022)
- [Enhancing Traffic Signal Control through Model-based Reinforcement Learning and Policy Reuse](https://mlsanity.com/paper/2503.08728/) (2025)
- [DSIP: A Dynamic Coordination Planner for Signal-Free Intersections using Diffusion-Model-Based Multi-Agent Motion Planning](https://mlsanity.com/paper/2606.30694/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) techniques for traffic signal control (TSC) have gained increasing popularity in recent years. However, most existing RL-based TSC methods tend to focus primarily on the RL model structure while neglecting the significance of proper traffic state representation. Furthermore, some RL-based methods heavily rely on expert-designed traffic signal phase competition. In this paper, we present a novel approach to TSC that utilizes queue length as an efficient state representation. We propose two new methods: (1) Max Queue-Length (M-QL), an optimization-based traditional method designed based on the property of queue length; and (2) AttentionLight, an RL model that employs the self-attention mechanism to capture the signal phase correlation without requiring human knowledge of phase relationships. Comprehensive experiments on multiple real-world datasets demonstrate the effectiveness of our approach: (1) the M-QL method outperforms the latest RL-based methods; (2) AttentionLight achieves a new state-of-the-art performance; and (3) our results highlight the significance of proper state representation, which is as crucial as neural network design in TSC methods. Our findings have important implications for advancing the development of more effective and efficient TSC methods. Our code is released on Github (https://github. com/LiangZhang1996/AttentionLight).