mlsanityData to 7 Oct 2026

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

arXiv 2407.08164 ↗First posted 11 Jul 2024cs.AI cs.MA cs.ROcited by 12

Not on the map. Why?

Abstract (arXiv)

In multi-agent reinforcement learning (MARL), the Centralized Training with Decentralized Execution (CTDE) framework is pivotal but struggles due to a gap: global state guidance in training versus reliance on local observations in execution, lacking global signals. Inspired by human societal consensus mechanisms, we introduce the Hierarchical Consensus-based Multi-Agent Reinforcement Learning (HC-MARL) framework to address this limitation. HC-MARL employs contrastive learning to foster a global consensus among agents, enabling cooperative behavior without direct communication. This approach enables agents to form a global consensus from local observations, using it as an additional piece of information to guide collaborative actions during execution. To cater to the dynamic requirements of various tasks, consensus is divided into multiple layers, encompassing both short-term and long-term considerations. Short-term observations prompt the creation of an immediate, low-layer consensus, while long-term observations contribute to the formation of a strategic, high-layer consensus. This process is further refined through an adaptive attention mechanism that dynamically adjusts the influence of each consensus layer. This mechanism optimizes the balance between immediate reactions and strategic planning, tailoring it to the specific demands of the task at hand. Extensive experiments and real-world applications in multi-robot systems showcase our framework's superior performance, marking significant advancements over baselines.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

27 of 43 references.

References · 27

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2020Supervised Contrastive Learningcited by 2,803
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  9. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  10. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  11. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  12. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
15 more
  1. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  2. 2020Social NCE: Contrastive Learning of Socially-aware Motion Representationscited by 58
  3. 2018Deep Reinforcement Learning for Swarm Systemscited by 43
  4. 2021A Physics-Informed Deep Learning Paradigm for Traffic State and Fundamental Diagram Estimationcited by 37
  5. 2018Learning Actionable Representations from Visual Observationscited by 35
  6. 2016Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Searchcited by 25
  7. 2020Learning Structured Communication for Multi-agent Reinforcement Learningcited by 21
  8. 2023Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Gamecited by 18
  9. 2020Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networkscited by 16
  10. 2023Leveraging Partial Symmetry for Multi-Agent Reinforcement Learningcited by 11
  11. 2023ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learningcited by 9
  12. 2023GraphRARE: Reinforcement Learning Enhanced Graph Neural Network with Relative Entropycited by 9
  13. 2022Consensus Learning for Cooperative Multi-Agent Reinforcement Learningcited by 8
  14. 2020Cooperative Multi-Agent Reinforcement Learning with Partial Observationscited by 6
  15. 2023Robust Multi-Agent Reinforcement Learning by Mutual Information Regularizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 3.

12 papers.

  1. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  2. 2025A Taxonomy of Hierarchical Multi-Agent Systems: Design Patterns, Coordination Mechanisms, and Industrial Applicationscited by 8
  3. 2026HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Executioncited by 5
  4. 2025SIGMA: Sheaf-Informed Geometric Multi-Agent Pathfindingcited by 4
  5. 2025EmbodiedAgent: A Scalable Hierarchical Approach to Overcome Practical Challenge in Multi-Robot Controlcited by 3
  6. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  7. 2025Spatial-Temporal Transformer with Curriculum Learning for EEG-Based Emotion Recognitioncited by 1
  8. 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learningcited by 1
  9. 2025Structured Diversity Control: A Dual-Level Framework for Group-Aware Multi-Agent Coordinationcited by 0
  10. 2025Symmetry-Guided Multi-Agent Inverse Reinforcement Learningcited by 0
  11. 2025HCPO: Hierarchical Conductor-Based Policy Optimization in Multi-Agent Reinforcement Learningcited by 0
  12. 2026Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuningcited by 0