# A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Control

arXiv: https://arxiv.org/abs/2503.14250 · HTML: https://mlsanity.com/paper/2503.14250/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 13 of 13)

- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [Reinforcement Learning with Parameterized Actions](https://mlsanity.com/paper/1509.01644/) (2015)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks](https://mlsanity.com/paper/1803.11115/) (2018)
- [Deep Deterministic Policy Gradient for Urban Traffic Light Control](https://mlsanity.com/paper/1703.09035/) (2017)
- [Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control](https://mlsanity.com/paper/2112.10107/) (2021)
- [Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces](https://mlsanity.com/paper/1905.04388/) (2019)
- [Efficient Pressure: Improving efficiency for signalized intersections](https://mlsanity.com/paper/2112.02336/) (2021)
- [DuaLight: Enhancing Traffic Signal Control by Leveraging Scenario-Specific and Scenario-Shared Knowledge](https://mlsanity.com/paper/2312.14532/) (2023)
- [DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback](https://mlsanity.com/paper/2306.07553/) (2023)
- [Reinforcement learning for traffic signal control in hybrid action space](https://mlsanity.com/paper/2211.12956/) (2022)
- [CycLight: learning traffic signal cooperation with a cycle-level strategy](https://mlsanity.com/paper/2401.08121/) (2024)

## Abstract (arXiv)

Adaptive traffic signal control (ATSC) can effectively reduce vehicle travel times by dynamically adjusting signal timings but poses a critical challenge in real-world scenarios due to the complexity of real-time decision-making in dynamic and uncertain traffic conditions. The burgeoning field of intelligent transportation systems, bolstered by artificial intelligence techniques and extensive data availability, offers new prospects for the implementation of ATSC. In this study, we introduce a parallel hybrid action space reinforcement learning model (PH-DDPG) that optimizes traffic signal phase and duration of traffic signals simultaneously, eliminating the need for sequential decision-making seen in traditional two-stage models. Our model features a task-specific parallel hybrid action space tailored for adaptive traffic control, which directly outputs discrete phase selections and their associated continuous duration parameters concurrently, thereby inherently addressing dynamic traffic adaptation through unified parametric optimization. %Our model features a unique parallel hybrid action space that allows for the simultaneous output of each action and its optimal parameters, streamlining the decision-making process. Furthermore, to ascertain the robustness and effectiveness of this approach, we executed ablation studies focusing on the utilization of a random action parameter mask within the critic network, which decouples the parameter space for individual actions, facilitating the use of preferable parameters for each action. The results from these studies confirm the efficacy of this method, distinctly enhancing real-world applicability