mlsanityData to 14 Oct 2026

Multi-hop Upstream Anticipatory Traffic Signal Control with Deep Reinforcement Learning

arXiv 2411.07271 ↗First posted 10 Nov 2024cs.LG cs.AI cs.SYcited by 2

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

Coordination in traffic signal control is crucial for managing congestion in urban networks. Existing pressure-based control methods focus only on immediate upstream links, leading to suboptimal green time allocation and increased network delays. However, effective signal control inherently requires coordination across a broader spatial scope, as the effect of upstream traffic should influence signal control decisions at downstream intersections, impacting a large area in the traffic network. Although agent communication using neural network-based feature extraction can implicitly enhance spatial awareness, it significantly increases the learning complexity, adding an additional layer of difficulty to the challenging task of control in deep reinforcement learning. To address the issue of learning complexity and myopic traffic pressure definition, our work introduces a novel concept based on Markov chain theory, namely textitmulti-hop upstream pressure, which generalizes the conventional pressure to account for traffic conditions beyond the immediate upstream links. This farsighted and compact metric informs the deep reinforcement learning agent to preemptively clear the multi-hop upstream queues, guiding the agent to optimize signal timings with a broader spatial awareness. Simulations on synthetic and realistic (Toronto) scenarios demonstrate controllers utilizing multi-hop upstream pressure significantly reduce overall network delay by prioritizing traffic movements based on a broader understanding of upstream congestion.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 48 references that are in the map. First posted on arXiv 10 Nov 2024.

Builds on

11 of 48 references.

References · 11

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 138
  3. 2022Two-layer adaptive signal control framework for large-scale dynamically-congested networks: Combining efficient Max Pressure with Perimeter Controlcited by 12
  4. 2022The Impact of Task Underspecification in Evaluating Deep Reinforcement Learningcited by 10
  5. 2013Decentralized Signal Control for Urban Road Networkscited by 7
  6. 2022Model-based graph reinforcement learning for inductive traffic signal controlcited by 5
  7. 2023Improving the generalizability and robustness of large-scale traffic signal controlcited by 5
  8. 2022A Novel Max Pressure Algorithm Based on Traffic Delaycited by 3
  9. 2024OCC-MP: A Max-Pressure framework to prioritize transit and high occupancy vehiclescited by 1
  10. 2024C-MP: A decentralized adaptive-coordinated traffic signal control using the Max Pressure frameworkcited by 1
  11. 2024Generalized Multi-hop Traffic Pressure for Heterogeneous Traffic Perimeter Controlcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Beyond 9-to-5: A Generative Model for Augmenting Mobility Data of Underrepresented Shift Workerscited by 1
  2. 2026Explainable Reinforcement Learning for Adaptive Traffic Signal Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Communication Strategy on Macro-and-Micro Traffic State in Cooperative Deep Reinforcement Learning for Regional Traffic Signal Control
  2. 2025Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning
  3. 2024Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion
  4. 2025Robust Single-Agent Reinforcement Learning for Regional Traffic Signal Control Under Demand Fluctuations
  5. 2025A Hierarchical Deep Reinforcement Learning Framework for Traffic Signal Control with Predictable Cycle Planning
  6. 2024Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs
  7. 2023Traffic Shaping and Hysteresis Mitigation Using Deep Reinforcement Learning in a Connected Driving Environment
  8. 2025Self-Regulating Cars: Automating Traffic Control in Free Flow Road Networks