mlsanityData to 7 Oct 2026

Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machines

arXiv 2403.07005 ↗First posted 8 Mar 2024cs.AI cs.LG cs.MAcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the cooperative Multi-Agent Reinforcement Learning (MARL) problems using Reward Machines (RMs) to specify the reward functions such that the prior knowledge of high-level events in a task can be leveraged to facilitate the learning efficiency. Unlike the existing work that RMs have been incorporated into MARL for task decomposition and policy learning in relatively simple domains or with an assumption of independencies among the agents, we present Multi-Agent Reinforcement Learning with a Hierarchy of RMs (MAHRM) that is capable of dealing with more complex scenarios when the events among agents can occur concurrently and the agents are highly interdependent. MAHRM exploits the relationship of high-level events to decompose a task into a hierarchy of simpler subtasks that are assigned to a small group of agents, so as to reduce the overall computational complexity. Experimental results in three cooperative MARL domains show that MAHRM outperforms other MARL methods using the same prior knowledge of high-level events.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

23 of 39 references.

References · 23

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  3. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  4. 2016The Option-Critic Architecturecited by 433
  5. 2016Learning Multiagent Communication with Backpropagationcited by 355
  6. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  7. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  8. 2017A Survey of Learning in Multiagent Environments: Dealing with Non-Stationaritycited by 93
  9. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 90
  10. 2019Influence-Based Multi-Agent Explorationcited by 75
  11. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 68
  12. 2019Reinforcement Learning with Non-Markovian Rewardscited by 45
11 more
  1. 2019Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discoverycited by 42
  2. 2020Reward Machines for Cooperative Multi-Agent Reinforcement Learningcited by 35
  3. 2019Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learningcited by 32
  4. 2020Active Finite Reward Automaton Inference and Reinforcement Learning Using Queries and Counterexamplescited by 23
  5. 2020Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learningcited by 21
  6. 2019Option-Critic in Cooperative Multi-agent Systemscited by 17
  7. 2019Induction of Subgoal Automata for Reinforcement Learningcited by 17
  8. 2020Learning Non-Markovian Reward Models in MDPscited by 11
  9. 2020Extended Markov Games to Learn Multiple Tasks in Multi-Agent Reinforcement Learningcited by 9
  10. 2021Decentralized Graph-Based Multi-Agent Reinforcement Learning Using Reward Machinescited by 6
  11. 2020Disentangled Planning and Control in Vision Based Robotics via Reward Machinescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Efficient Reinforcement Learning in Probabilistic Reward Machinescited by 4
  2. 2025TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learningcited by 2
  3. 2026Communication Methods in Multi-Agent Reinforcement Learningcited by 1