mlsanityData to 7 Oct 2026

Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation

arXiv 2302.04094 ↗First posted 8 Feb 2023cs.RO cs.AI cs.MAcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper investigates the multi-agent navigation problem, which requires multiple agents to reach the target goals in a limited time. Multi-agent reinforcement learning (MARL) has shown promising results for solving this issue. However, it is inefficient for MARL to directly explore the (nearly) optimal policy in the large search space, which is exacerbated as the agent number increases (e.g., 10+ agents) or the environment is more complex (e.g., 3D simulator). Goal-conditioned hierarchical reinforcement learning (HRL) provides a promising direction to tackle this challenge by introducing a hierarchical structure to decompose the search space, where the low-level policy predicts primitive actions in the guidance of the goals derived from the high-level policy. In this paper, we propose Multi-Agent Graph-Enhanced Commander-Executor (MAGE-X), a graph-based goal-conditioned hierarchical method for multi-agent navigation tasks. MAGE-X comprises a high-level Goal Commander and a low-level Action Executor. The Goal Commander predicts the probability distribution of goals and leverages them to assign each agent the most appropriate final target. The Action Executor utilizes graph neural networks (GNN) to construct a subgraph for each agent that only contains crucial partners to improve cooperation. Additionally, the Goal Encoder in the Action Executor captures the relationship between the agent and the designated goal to encourage the agent to reach the final target. The results show that MAGE-X outperforms the state-of-the-art MARL baselines with a 100% success rate with only 3 million training steps in multi-agent particle environments (MPE) with 50 agents, and at least a 12% higher success rate and 2x higher data efficiency in a more complicated quadrotor 3D navigation task.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 27 of its 47 references that are in the map. First posted on arXiv 8 Feb 2023.

Builds on

27 of 47 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  9. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  10. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  11. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  12. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
15 more
  1. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  2. 2018Graph Convolutional Reinforcement Learningcited by 148
  3. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  4. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  5. 2021Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Controlcited by 68
  6. 2020Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictorscited by 60
  7. 2020Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learningcited by 58
  8. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 55
  9. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  10. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  11. 2019Multi-Agent Actor-Critic with Hierarchical Graph Attention Networkcited by 33
  12. 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
  13. 2022GCS: Graph-based Coordination Strategy for Multi-Agent Reinforcement Learningcited by 20
  14. 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
  15. 2019Inter-Level Cooperation in Hierarchical Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 2.

6 papers.

  1. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  2. 2024Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learningcited by 4
  3. 2023Active Neural Topological Mapping for Multi-Agent Explorationcited by 0
  4. 2023MASP: Scalable GNN-based Planning for Multi-Agent Navigationcited by 0
  5. 2026COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systemscited by 0
  6. 2026A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communicationcited by 0