# CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehicles

arXiv: https://arxiv.org/abs/2104.10340 · HTML: https://mlsanity.com/paper/2104.10340/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 63 references that are in the map.
- First posted on arXiv 21 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning](https://mlsanity.com/paper/2007.05156/) (2020)
- [Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2002.06723/) (2020)
- [Using Reinforcement Learning with Partial Vehicle Detection for Intelligent Traffic Signal Control](https://mlsanity.com/paper/1807.01628/) (2018)
- [Multi-Agent Reinforcement Learning for Markov Routing Games: A New Modeling Paradigm For Dynamic Traffic Assignment](https://mlsanity.com/paper/2011.10915/) (2020)
- [An Open-Source Framework for Adaptive Traffic Signal Control](https://mlsanity.com/paper/1909.00395/) (2019)
- [Efficiency and Equity are Both Essential: A Generalized Traffic Signal Controller with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.04046/) (2020)
- [Backpressure Control with Estimated Queue Lengths for Urban Network Traffic](https://mlsanity.com/paper/2006.15549/) (2020)
- [Cooperative Schedule-Driven Intersection Control with Connected and Autonomous Vehicles](https://mlsanity.com/paper/1907.01984/) (2019)
- [Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication](https://mlsanity.com/paper/2002.09853/) (2020)

## Built on by (most cited first, 11 of 11)

- [An Open and Large-Scale Dataset for Multi-Modal Climate Change-aware Crop Yield Predictions](https://mlsanity.com/paper/2406.06081/) (2024)
- [The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality](https://mlsanity.com/paper/2206.11996/) (2022)
- [EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System](https://mlsanity.com/paper/2206.13441/) (2022)
- [Reinforcement learning for traffic signal control in hybrid action space](https://mlsanity.com/paper/2211.12956/) (2022)
- [A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning](https://mlsanity.com/paper/2311.15920/) (2023)
- [AI-Powered CPS-Enabled Vulnerable-User-Aware Urban Transportation Digital Twin: Methods and Applications](https://mlsanity.com/paper/2501.10396/) (2024)
- [Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control](https://mlsanity.com/paper/2504.05553/) (2025)
- [Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning](https://mlsanity.com/paper/2210.02612/) (2022)
- [Analyzing Multi-Head Attention on Trojan BERT Models](https://mlsanity.com/paper/2406.16925/) (2024)
- [The Streetscape Application Services Stack (SASS): Towards a Distributed Sensing Architecture for Urban Applications](https://mlsanity.com/paper/2411.19714/) (2024)
- [Reinforcement Learning-Based Traffic Signal Control for IoT-Enabled Intersections](https://mlsanity.com/paper/2606.22108/) (2026)

## Abstract (arXiv)

This paper develops a decentralized reinforcement learning (RL) scheme for multi-intersection adaptive traffic signal control (TSC), called "CVLight", that leverages data collected from connected vehicles (CVs). The state and reward design facilitates coordination among agents and considers travel delays collected by CVs. A novel algorithm, Asymmetric Advantage Actor-critic (Asym-A2C), is proposed where both CV and non-CV information is used to train the critic network, while only CV information is used to execute optimal signal timing. Comprehensive experiments show the superiority of CVLight over state-of-the-art algorithms under a 2-by-2 synthetic road network with various traffic demand patterns and penetration rates. The learned policy is then visualized to further demonstrate the advantage of Asym-A2C. A pre-train technique is applied to improve the scalability of CVLight, which significantly shortens the training time and shows the advantage in performance under a 5-by-5 road network. A case study is performed on a 2-by-2 road network located in State College, Pennsylvania, USA, to further demonstrate the effectiveness of the proposed algorithm under real-world scenarios. Compared to other baseline models, the trained CVLight agent can efficiently control multiple intersections solely based on CV data and achieve the best performance, especially under low CV penetration rates.