mlsanityData to 14 Oct 2026

Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control

arXiv 2504.05553 ↗First posted 7 Apr 2025cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

Multi-agent reinforcement learning (MARL) has shown promise for adaptive traffic signal control (ATSC), enabling multiple intersections to coordinate signal timings in real time. However, in large-scale settings, MARL faces constraints due to extensive data sharing and communication requirements. Federated learning (FL) mitigates these challenges by training shared models without directly exchanging raw data, yet traditional FL methods such as FedAvg struggle with highly heterogeneous intersections. Different intersections exhibit varying traffic patterns, demands, and road structures, so performing FedAvg across all agents is inefficient. To address this gap, we propose Hierarchical Federated Reinforcement Learning (HFRL) for ATSC. HFRL employs clustering-based or optimization-based techniques to dynamically group intersections and perform FedAvg independently within groups of intersections with similar characteristics, enabling more effective coordination and scalability than standard FedAvg.Our experiments on synthetic and real-world traffic networks demonstrate that HFRL consistently outperforms decentralized and standard federated RL approaches, and achieves competitive or superior performance compared to centralized RL as network scale and heterogeneity increase, particularly in real-world settings. The method also identifies suitable grouping patterns based on network structure or traffic demand, resulting in a more robust framework for distributed, heterogeneous systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 63 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

23 of 63 references.

References · 23

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  4. 2019Federated Learning with Personalization Layerscited by 541
  5. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  6. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  7. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  8. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  9. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  10. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  11. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  12. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
11 more
  1. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  2. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  3. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 90
  4. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  5. 2021Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learningcited by 53
  6. 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
  7. 2019Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learningcited by 33
  8. 2022FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergencecited by 29
  9. 2021CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehiclescited by 11
  10. 2020Multi-Agent Reinforcement Learning for Markov Routing Games: A New Modeling Paradigm For Dynamic Traffic Assignmentcited by 9
  11. 2022FedFormer: Contextual Federation with Attention in Reinforcement Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Realistic Urban Traffic Generator using Decentralized Federated Learning for the SUMO simulatorcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Toward Dependency Dynamics in Multi-Agent Reinforcement Learning for Traffic Signal Control
  2. 2024Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion
  3. 2025A Semi Centralized Training Decentralized Execution Architecture for Multi Agent Deep Reinforcement Learning in Traffic Signal Control
  4. 2025A Hierarchical Signal Coordination and Control System Using a Hybrid Model-based and Reinforcement Learning Approach
  5. 2025A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Control
  6. 2025HALO: Hierarchical Reinforcement Learning for Large-Scale Adaptive Traffic Signal Control
  7. 2025Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning
  8. 2026CROSS: A Mixture-of-Experts Reinforcement Learning Framework for Generalizable Large-Scale Traffic Signal Control