A Hierarchical Deep Reinforcement Learning Framework for Traffic Signal Control with Predictable Cycle Planning
On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic
Abstract (arXiv)
Deep reinforcement learning (DRL) has become a popular approach in traffic signal control (TSC) due to its ability to learn adaptive policies from complex traffic environments. Within DRL-based TSC methods, two primary control paradigms are ``choose phase" and ``switch" strategies. Although the agent in the choose phase paradigm selects the next active phase adaptively, this paradigm may result in unexpected phase sequences for drivers, disrupting their anticipation and potentially compromising safety at intersections. Meanwhile, the switch paradigm allows the agent to decide whether to switch to the next predefined phase or extend the current phase. While this structure maintains a more predictable order, it can lead to unfair and inefficient phase allocations, as certain movements may be extended disproportionately while others are neglected. In this paper, we propose a DRL model, named Deep Hierarchical Cycle Planner (DHCP), to allocate the traffic signal cycle duration hierarchically. A high-level agent first determines the split of the total cycle time between the North-South (NS) and East-West (EW) directions based on the overall traffic state. Then, a low-level agent further divides the allocated duration within each major direction between straight and left-turn movements, enabling more flexible durations for the two movements. We test our model on both real and synthetic road networks, along with multiple sets of real and synthetic traffic flows. Empirical results show our model achieves the best performance over all datasets against baselines.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 30 references that are in the map. First posted on arXiv 3 Sep 2025.
Builds on
11 of 30 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
- 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
- 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 138
- 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 115
- 2020Deep Reinforcement Learning for Intelligent Transportation Systems: A Surveycited by 98
- 2019A Survey on Traffic Signal Control Methodscited by 79
- 2021Efficient Pressure: Improving efficiency for signalized intersectionscited by 24
- 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
- 2023Large-Scale Traffic Signal Control by a Nash Deep Q-network Approachcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Learning Traffic Signal Control via Genetic Programming
- 2024Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion
- 2025Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learning
- 2023A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning
- 2025Evolutionary Discovery of Heuristic Policies for Traffic Signal Control
- 2024Optimizing Traffic Signal Control using High-Dimensional State Representation and Efficient Deep Reinforcement Learning
- 2024Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning
- 2025A Hierarchical Signal Coordination and Control System Using a Hybrid Model-based and Reinforcement Learning Approach