mlsanityData to 7 Oct 2026

Hierarchical Lead Critic based Multi-Agent Reinforcement Learning

arXiv 2602.21680 ↗First posted 25 Feb 2026cs.LG cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

Cooperative Multi-Agent Reinforcement Learning (MARL) solves complex tasks that require coordination from multiple agents, but is often limited to either local (independent learning) or global (centralized learning) perspectives. In this paper, we introduce a novel sequential training scheme and MARL architecture, which learns from multiple perspectives on different hierarchy levels. We propose the Hierarchical Lead Critic (HLC) - inspired by natural emerging distributions in team structures, where following high-level objectives combines with low-level execution. HLC demonstrates that introducing multiple hierarchies, leveraging local and global perspectives, can lead to improved performance with high sample efficiency and robust policies. Experimental results conducted on cooperative, non-communicative, and partially observable MARL benchmarks demonstrate that HLC outperforms single hierarchy baselines and scales robustly with increasing amounts of agents and difficulty.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

21 of 32 references.

References · 21

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  9. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  10. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  11. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  12. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
9 more
  1. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  2. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  3. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
  4. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  5. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  6. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
  7. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  8. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  9. 2024SLIM: Skill Learning with Multiple Criticscited by 3

Built on by

Not cited yet.