# Multi-hop Upstream Anticipatory Traffic Signal Control with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2411.07271 · HTML: https://mlsanity.com/paper/2411.07271/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 48 references that are in the map.
- First posted on arXiv 10 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [Two-layer adaptive signal control framework for large-scale dynamically-congested networks: Combining efficient Max Pressure with Perimeter Control](https://mlsanity.com/paper/2210.10453/) (2022)
- [The Impact of Task Underspecification in Evaluating Deep Reinforcement Learning](https://mlsanity.com/paper/2210.08607/) (2022)
- [Decentralized Signal Control for Urban Road Networks](https://mlsanity.com/paper/1310.0491/) (2013)
- [Model-based graph reinforcement learning for inductive traffic signal control](https://mlsanity.com/paper/2208.00659/) (2022)
- [Improving the generalizability and robustness of large-scale traffic signal control](https://mlsanity.com/paper/2306.01925/) (2023)
- [A Novel Max Pressure Algorithm Based on Traffic Delay](https://mlsanity.com/paper/2202.03290/) (2022)
- [OCC-MP: A Max-Pressure framework to prioritize transit and high occupancy vehicles](https://mlsanity.com/paper/2406.19269/) (2024)
- [C-MP: A decentralized adaptive-coordinated traffic signal control using the Max Pressure framework](https://mlsanity.com/paper/2407.01421/) (2024)
- [Generalized Multi-hop Traffic Pressure for Heterogeneous Traffic Perimeter Control](https://mlsanity.com/paper/2409.00753/) (2024)

## Built on by (most cited first, 2 of 2)

- [Beyond 9-to-5: A Generative Model for Augmenting Mobility Data of Underrepresented Shift Workers](https://mlsanity.com/paper/2507.19510/) (2025)
- [Explainable Reinforcement Learning for Adaptive Traffic Signal Control](https://mlsanity.com/paper/2607.03703/) (2026)

## Abstract (arXiv)

Coordination in traffic signal control is crucial for managing congestion in urban networks. Existing pressure-based control methods focus only on immediate upstream links, leading to suboptimal green time allocation and increased network delays. However, effective signal control inherently requires coordination across a broader spatial scope, as the effect of upstream traffic should influence signal control decisions at downstream intersections, impacting a large area in the traffic network. Although agent communication using neural network-based feature extraction can implicitly enhance spatial awareness, it significantly increases the learning complexity, adding an additional layer of difficulty to the challenging task of control in deep reinforcement learning. To address the issue of learning complexity and myopic traffic pressure definition, our work introduces a novel concept based on Markov chain theory, namely textitmulti-hop upstream pressure, which generalizes the conventional pressure to account for traffic conditions beyond the immediate upstream links. This farsighted and compact metric informs the deep reinforcement learning agent to preemptively clear the multi-hop upstream queues, guiding the agent to optimize signal timings with a broader spatial awareness. Simulations on synthetic and realistic (Toronto) scenarios demonstrate controllers utilizing multi-hop upstream pressure significantly reduce overall network delay by prioritizing traffic movements based on a broader understanding of upstream congestion.