# DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback

arXiv: https://arxiv.org/abs/2306.07553 · HTML: https://mlsanity.com/paper/2306.07553/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 81 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Tianshou: a Highly Modularized Deep Reinforcement Learning Library](https://mlsanity.com/paper/2107.14171/) (2021)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering](https://mlsanity.com/paper/2207.12647/) (2022)
- [Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition](https://mlsanity.com/paper/2009.00210/) (2020)
- [Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1908.03761/) (2019)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [Efficient Pressure: Improving efficiency for signalized intersections](https://mlsanity.com/paper/2112.02336/) (2021)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Deep Image-to-Video Adaptation and Fusion Networks for Action Recognition](https://mlsanity.com/paper/1911.10751/) (2019)
- [Hierarchically Learned View-Invariant Representations for Cross-View Action Recognition](https://mlsanity.com/paper/1809.00421/) (2018)
- [Global Temporal Representation based CNNs for Infrared Action Recognition](https://mlsanity.com/paper/1909.08287/) (2019)
- [Urban Regional Function Guided Traffic Flow Prediction](https://mlsanity.com/paper/2303.09789/) (2023)

## Built on by (most cited first, 7 of 7)

- [CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal Reasoning](https://mlsanity.com/paper/2306.17462/) (2023)
- [MEIA: Multimodal Embodied Perception and Interaction in Unknown Environments](https://mlsanity.com/paper/2402.00290/) (2024)
- [ODMixer: Fine-grained Spatial-temporal MLP for Metro Origin-Destination Prediction](https://mlsanity.com/paper/2404.15734/) (2024)
- [CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2603.24366/) (2026)
- [LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2603.24361/) (2026)
- [A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Control](https://mlsanity.com/paper/2503.14250/) (2025)
- [HALO: Hierarchical Reinforcement Learning for Large-Scale Adaptive Traffic Signal Control](https://mlsanity.com/paper/2506.14391/) (2025)

## Abstract (arXiv)

Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network, which in turn enhances fuel utilization efficiency, air quality, and road safety, benefiting society as a whole. Due to the complexity of long-horizon control and coordination, most prior TSC methods leverage deep reinforcement learning (RL) to search for a control policy and have witnessed great success. However, TSC still faces two significant challenges. 1) The travel time of a vehicle is delayed feedback on the effectiveness of TSC policy at each traffic intersection since it is obtained after the vehicle has left the road network. Although several heuristic reward functions have been proposed as substitutes for travel time, they are usually biased and not leading the policy to improve in the correct direction. 2) The traffic condition of each intersection is influenced by the non-local intersections since vehicles traverse multiple intersections over time. Therefore, the TSC agent is required to leverage both the local observation and the non-local traffic conditions to predict the long-horizontal traffic conditions of each intersection comprehensively. To address these challenges, we propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness and a non-local enhanced TSC agent to better predict future traffic conditions for more precise traffic control. Extensive experiments and ablation studies demonstrate that DenseLight can consistently outperform advanced baselines on various road networks with diverse traffic flows. The code is available at https://github.com/junfanlin/DenseLight.