mlsanityData to 14 Oct 2026

Using a single actor to output personalized policy for different intersections

arXiv 2503.07678 ↗First posted 10 Mar 2025cs.LG cs.AI cs.MAcited by 0

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

Recently, with the development of Multi-agent reinforcement learning (MARL), adaptive traffic signal control (ATSC) has achieved satisfactory results. In traffic scenarios with multiple intersections, MARL treats each intersection as an agent and optimizes traffic signal control strategies through learning and real-time decision-making. Considering that observation distributions of intersections might be different in real-world scenarios, shared parameter methods might lack diversity and thus lead to high generalization requirements in the shared-policy network. A typical solution is to increase the size of network parameters. However, simply increasing the scale of the network does not necessarily improve policy generalization, which is validated in our experiments. Accordingly, an approach that considers both the personalization of intersections and the efficiency of parameter sharing is required. To this end, we propose Hyper-Action Multi-Head Proximal Policy Optimization (HAMH-PPO), a Centralized Training with Decentralized Execution (CTDE) MARL method that utilizes a shared PPO policy network to deliver personalized policies for intersections with non-iid observation distributions. The centralized critic in HAMH-PPO uses graph attention units to calculate the graph representations of all intersections and outputs a set of value estimates with multiple output heads for each intersection. The decentralized execution actor takes the local observation history as input and output distributions of action as well as a so-called hyper-action to balance the multiple values estimated from the centralized critic to further guide the updating of TSC policies. The combination of hyper-action and multi-head values enables multiple agents to share a single actor-critic while achieving personalized policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 37 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

14 of 37 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Graph Attention Networkscited by 8,923
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2019Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcited by 164
  7. 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 138
  8. 2019Learning Phase Competition for Traffic Signal Controlcited by 87
  9. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
  10. 2021Efficient Pressure: Improving efficiency for signalized intersectionscited by 24
  11. 2020IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Controlcited by 22
  12. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
2 more
  1. 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
  2. 2021Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problemcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Semi Centralized Training Decentralized Execution Architecture for Multi Agent Deep Reinforcement Learning in Traffic Signal Control
  2. 2025A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Control
  3. 2025Smart Traffic Signals: Comparing MARL and Fixed-Time Strategies
  4. 2024Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs
  5. 2025Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control
  6. 2025Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control
  7. 2025Toward Dependency Dynamics in Multi-Agent Reinforcement Learning for Traffic Signal Control
  8. 2025Convergence of Multiagent Learning Systems for Traffic control