# Offline multi-agent RL

Area: [Offline RL and imitation learning from data](https://mlsanity.com/offlinerl/index.md). HTML: https://mlsanity.com/offlinerl/offline-multi-agent-rl/

## Key facts

- Offline multi-agent RL: 59 papers on arXiv since 2021 (data through 14 Oct 2026).
- 18 of its 59 mapped papers (31%) cite “Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning” (2021, arXiv 2106.03400).
- 16 of its 59 mapped papers (27%) cite “Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification” (2021, arXiv 2111.11188).
- 2 new papers 17 Sep–14 Oct.

## Most cited here

- [Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.03400/) (2021), cited by 18 papers here
- [Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification](https://mlsanity.com/paper/2111.11188/) (2021), cited by 16 papers here
- [A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem](https://mlsanity.com/paper/2305.17198/) (2023), cited by 7 papers here
- [MADiff: Offline Multi-agent Learning with Diffusion Models](https://mlsanity.com/paper/2305.17330/) (2023), cited by 7 papers here
- [Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization](https://mlsanity.com/paper/2307.11620/) (2023), cited by 7 papers here
- [Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets](https://mlsanity.com/paper/2311.11423/) (2023), cited by 7 papers here
- [AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation](https://mlsanity.com/paper/2311.02194/) (2023), cited by 5 papers here
- [Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL](https://mlsanity.com/paper/2307.01472/) (2023), cited by 4 papers here
- [Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.08421/) (2024), cited by 4 papers here
- [Coordination Failure in Cooperative Offline MARL](https://mlsanity.com/paper/2407.01343/) (2024), cited by 4 papers here

## New (17 Sep–14 Oct)

- [Test-time Multi-agent Coordination by Decomposed Value Gradient Flow](https://mlsanity.com/paper/2610.02554/) (2026-10-01)
- [G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies](https://mlsanity.com/paper/2609.31286/) (2026-09-25)