mlsanityData to 7 Oct 2026

Offline multi-agent RL

59 papers0 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: multi-agent, marl, offline, offline multi-agent, multi-agent reinforcement

Key facts. Offline multi-agent RL: 59 papers on arXiv since 2021 (data through 7 Oct 2026). 18 of its 59 mapped papers (31%) cite “Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning” (2021, arXiv 2106.03400). 16 of its 59 mapped papers (27%) cite “Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification” (2021, arXiv 2111.11188). 2 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

2 new since 17 Sep

  1. Test-time Multi-agent Coordination by Decomposed Value Gradient Flow
  2. G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies

Foundations

Most cited by papers here

  1. Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning2021cited by 18 here
  2. Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification2021cited by 16 here
  3. A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem2023cited by 7 here
  4. MADiff: Offline Multi-agent Learning with Diffusion Models2023cited by 7 here
  5. Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization2023cited by 7 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2019An Optimistic Perspective on Offline Reinforcement Learningcited by 7 here
  2. 2019Behavior Regularized Offline Reinforcement Learningcited by 9 here
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 12 here
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 25 here
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 28 here
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 11 here
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 9 here
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 17 here
  9. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 16 here
  10. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 7 here
  11. 2024Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learningcited by 4 here
13 more it builds on, not in this chain
  1. 2020MOPO: Model-based Offline Policy Optimizationcited by 6 here
  2. 2020Critic Regularized Regressioncited by 4 here
  3. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 18 here
  4. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 4 here
  5. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 6 here
  6. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 5 here
  7. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 7 here
  8. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 4 here
  9. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 7 here
  10. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 6 here
  11. 2023AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimationcited by 5 here
  12. 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasetscited by 7 here
  13. 2024Coordination Failure in Cooperative Offline MARLcited by 4 here

The 24 most cited of 154.

Browse

30 of 59, ranked 6–35

  1. 6Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets20237
  2. 7AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation20235
  3. 8Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning20244
  4. 9Coordination Failure in Cooperative Offline MARL20244
  5. 10Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL20234
  6. 11An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management20253
  7. 12Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation20243
  8. 13Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization20243
  9. 14Learning from Good Trajectories in Offline Multi-Agent Reinforcement Learning20223
  10. 15ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization20242
All 30
  1. 16Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation20242
  2. 17Offline Multi-Agent Reinforcement Learning with Coupled Value Factorization20232
  3. 18Advancing RAN Slicing with Offline Reinforcement Learning20232
  4. 19Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning20251
  5. 20Offline Reinforcement Learning for Mobility Robustness Optimization20251
  6. 21OM2P: Offline Multi-Agent Mean-Flow Policy20251
  7. 22Diffusion-based Episodes Augmentation for Offline Multi-Agent Reinforcement Learning20241
  8. 23Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning20241
  9. 24Offline-to-Online Multi-Agent Reinforcement Learning with Offline Value Function Memory and Sequential Exploration20241
  10. 25Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learning20231
  11. 26Selective Reincarnation: Offline-to-Online Multi-Agent Reinforcement Learning20231
  12. 27Offline to Online Learning for Real-Time Bandwidth Estimation20231
  13. 28MACCA: Offline Multi-agent Reinforcement Learning with Causal Credit Assignment20231
  14. 29Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks20211
  15. 30Offline Multi-Agent Reinforcement Learning for 6G Communications: Fundamentals, Applications and Future Directions20260
  16. 31Puzzle it Out: Local-to-Global World Model for Offline Multi-Agent Reinforcement Learning20260
  17. 32Hybrid Orchestration of Edge AI and Microservices via Graph-based Self-Imitation Learning20260
  18. 33Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control20260
  19. 34A Recipe for Stable Offline Multi-agent Reinforcement Learning20260
  20. 35Learning Partial Action Replacement in Offline MARL20260