mlsanityData to 14 Oct 2026

Reinforcement learning for traffic signal control in hybrid action space

arXiv 2211.12956 ↗First posted 23 Nov 2022eess.SY cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

The prevailing reinforcement-learning-based traffic signal control methods are typically staging-optimizable or duration-optimizable, depending on the action spaces. In this paper, we propose a novel control architecture, TBO, which is based on hybrid proximal policy optimization. To the best of our knowledge, TBO is the first RL-based algorithm to implement synchronous optimization of the staging and duration. Compared to discrete and continuous action spaces, hybrid action space is a merged search space, in which TBO better implements the trade-off between frequent switching and unsaturated release. Experiments are given to demonstrate that TBO reduces the queue length and delay by 13.78% and 14.08% on average, respectively, compared to the existing baselines. Furthermore, we calculate the Gini coefficients of the right-of-way to indicate TBO does not harm fairness while improving efficiency.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 29 of its 76 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

29 of 76 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  9. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  10. 2018Distributed Prioritized Experience Replaycited by 269
  11. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
  12. 2019Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcited by 164
17 more
  1. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 162
  2. 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 138
  3. 2019Learning Phase Competition for Traffic Signal Controlcited by 87
  4. 2015Reinforcement Learning with Parameterized Actionscited by 63
  5. 2020AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Controlcited by 62
  6. 2018Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Spacecited by 58
  7. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 54
  8. 2018Deep Reinforcement Learning for Traffic Light Control in Vehicular Networkscited by 52
  9. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 35
  10. 2017Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learningcited by 33
  11. 2017Deep Deterministic Policy Gradient for Urban Traffic Light Controlcited by 30
  12. 2019Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spacescited by 24
  13. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 22
  14. 2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 18
  15. 2021CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehiclescited by 11
  16. 2018Using Reinforcement Learning with Partial Vehicle Detection for Intelligent Traffic Signal Controlcited by 10
  17. 2021Policy Regularization via Noisy Advantage Values for Cooperative Multi-agent Actor-Critic methodscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2024DRL-Based Optimization for AoI and Energy Consumption in C-V2X Enabled IoVcited by 6
  2. 2026Human-Centric Traffic Signal Control for Equity: A Multi-Agent Action Branching Deep Reinforcement Learning Approachcited by 1
  3. 2025A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Controlcited by 0
  4. 2026Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal controlcited by 0
  5. 2026Temporal Action Representation Learning for Tactical Resource Control and Subsequent Maneuver Generationcited by 0