mlsanityData to 7 Oct 2026

Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning

arXiv 1901.08492 ↗First posted 24 Jan 2019cs.MA cs.AI cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

We investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from human societies, in which successful coordination of many individuals is often facilitated by hierarchical organisation, we introduce Feudal Multi-agent Hierarchies (FMH). In this framework, a 'manager' agent, which is tasked with maximising the environmentally-determined reward function, learns to communicate subgoals to multiple, simultaneously-operating, 'worker' agents. Workers, which are rewarded for achieving managerial subgoals, take concurrent actions in the world. We outline the structure of FMH and demonstrate its potential for decentralised learning and control. We find that, given an adequate set of subgoals from which to choose, FMH performs, and particularly scales, substantially better than cooperative approaches that use a shared reward function.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 2; 2024: 5; 2025: 8; 2026: 5.

32 papers.

Sort:

By area:

  1. 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 119
  2. 2022Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workerscited by 28
  3. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  4. 2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 17
  5. 2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 16
  6. 2024AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Frameworkcited by 12
  7. 2022Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantizationcited by 11
  8. 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushingcited by 9
  9. 2025A Taxonomy of Hierarchical Multi-Agent Systems: Design Patterns, Coordination Mechanisms, and Industrial Applicationscited by 8
  10. 2025Predicting Multi-Agent Specialization via Task Parallelizabilitycited by 5
  11. 2026HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Executioncited by 5
  12. 2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 4
20 more
  1. 2021Learning to Guide and to Be Guided in the Architect-Builder Problemcited by 3
  2. 2023Feudal Graph Reinforcement Learningcited by 3
  3. 2024Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machinescited by 3
  4. 2025GraphicBench: A Planning Benchmark for Graphic Design with Language Agentscited by 3
  5. 2026RUMAD: Reinforcement-Unifying Multi-Agent Debatecited by 3
  6. 2022Solving Dynamic Principal-Agent Problems with a Rationally Inattentive Principalcited by 2
  7. 2022Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observabilitycited by 2
  8. 2023Vehicle Dispatching and Routing of On-Demand Intercity Ride-Pooling Services: A Multi-Agent Hierarchical Reinforcement Learning Approachcited by 2
  9. 2025TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learningcited by 2
  10. 2025Implicitly Aligning Humans and Autonomous Agents through Shared Task Abstractionscited by 2
  11. 2021Scalable, Decentralized Multi-Agent Reinforcement Learning Methods Inspired by Stigmergy and Ant Coloniescited by 1
  12. 2022Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperationcited by 1
  13. 2025Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learningcited by 1
  14. 2021Room Clearance with Feudal Hierarchical Reinforcement Learningcited by 0
  15. 2022Hierarchically Structured Scheduling and Execution of Tasks in a Multi-Agent Environmentcited by 0
  16. 2024Effective Reward Specification in Deep Reinforcement Learningcited by 0
  17. 2025Zero-Incentive Dynamics: a look at reward sparsity through the lens of unrewarded subgoalscited by 0
  18. 2026Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Marketscited by 0
  19. 2026Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Controlcited by 0
  20. 2026HiComm: Hierarchical Communication for Multi-agent Reinforcement Learningcited by 0