mlsanityData to 7 Oct 2026

Option-Critic in Cooperative Multi-agent Systems

arXiv 1911.12825 ↗First posted 28 Nov 2019cs.AI cs.MA cs.SYcited by 17

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate learning temporal abstractions in cooperative multi-agent systems, using the options framework (Sutton et al, 1999). First, we address the planning problem for the decentralized POMDP represented by the multi-agent system, by introducing a common information approach. We use the notion of common beliefs and broadcasting to solve an equivalent centralized POMDP problem. Then, we propose the Distributed Option Critic (DOC) algorithm, which uses centralized option evaluation and decentralized intra-option improvement. We theoretically analyze the asymptotic convergence of DOC and build a new multi-agent environment to demonstrate its validity. Our experiments empirically show that DOC performs competitively against baselines and scales with the number of agents.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 6; 2023: 2; 2024: 5; 2025: 3.

17 papers.

Sort:

By area:

  1. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  2. 2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 16
  3. 2023Scalable Multi-agent Covering Option Discovery based on Kronecker Graphscited by 9
  4. 2023Optimizing delegation between human and AI collaborative agentscited by 9
  5. 2024On Centralized Critics in Multi-Agent Reinforcement Learningcited by 9
  6. 2024Variational Offline Multi-agent Skill Discoverycited by 6
  7. 2022Learning Multi-agent Skills for Tabular Reinforcement Learning using Factor Graphscited by 4
  8. 2024Optimizing Delegation in Collaborative Human-AI Hybrid Teamscited by 3
  9. 2024Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machinescited by 3
  10. 2022Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observabilitycited by 2
  11. 2022Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperationcited by 1
  12. 2024Optimizing Risk-averse Human-AI Hybrid Teamscited by 1
5 more
  1. 2025Comprehend, Divide, and Conquer: Feature Subspace Exploration via Multi-Agent Hierarchical Reinforcement Learningcited by 1
  2. 2022Coordinating Policies Among Multiple Agents via an Intelligent Communication Channelcited by 0
  3. 2022Multi-agent Deep Covering Skill Discoverycited by 0
  4. 2025Hierarchical Reinforcement Learning for Optimal Agent Grouping in Cooperative Systemscited by 0
  5. 2025Enhancing Multi-Agent Collaboration with Attention-Based Actor-Critic Policiescited by 0