mlsanityData to 7 Oct 2026

Diffusion Models for Offline Multi-agent Reinforcement Learning with Safety Constraints

arXiv 2407.00741 ↗First posted 30 Jun 2024cs.AI cs.LG cs.ROcited by 1

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

In recent advancements in Multi-agent Reinforcement Learning (MARL), its application has extended to various safety-critical scenarios. However, most methods focus on online learning, which presents substantial risks when deployed in real-world settings. Addressing this challenge, we introduce an innovative framework integrating diffusion models within the MARL paradigm. This approach notably enhances the safety of actions taken by multiple agents through risk mitigation while modeling coordinated action. Our framework is grounded in the Centralized Training with Decentralized Execution (CTDE) architecture, augmented by a Diffusion Model for prediction trajectory generation. Additionally, we incorporate a specialized algorithm to further ensure operational safety. We evaluate our model against baselines on the DSRL benchmark. Experiment results demonstrate that our model not only adheres to stringent safety constraints but also achieves superior performance compared to existing methodologies. This underscores the potential of our approach in advancing the safety and efficacy of MARL in real-world applications.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 38 of its 56 references that are in the map. First posted on arXiv 30 Jun 2024.

Builds on

38 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov

Other references · 37

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2015Continuous control with deep reinforcement learningcited by 3,279
  9. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  10. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  11. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  12. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
25 more
  1. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  2. 2017Constrained Policy Optimizationcited by 928
  3. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  4. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  5. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  7. 2017Deep Reinforcement Learning: An Overviewcited by 327
  8. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  9. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  11. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  12. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  13. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  14. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  15. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  16. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  17. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 180
  18. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
  19. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  20. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  21. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  22. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  23. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 55
  24. 2018Factorized Q-Learning for Large-Scale Multi-Agent Systemscited by 19
  25. 2020Diffusing-Horizon Model Predictive Controlcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Puzzle it Out: Local-to-Global World Model for Offline Multi-Agent Reinforcement Learningcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Individual Control Barrier Functions-Guided Diffusion Model for Safe Offline Multi-Agent Reinforcement Learning
  2. 2025Don't Trade Off Safety: Diffusion Regularization for Constrained Offline RL
  3. 2025Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Development
  4. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL
  5. 2024Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation
  6. 2025Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective
  7. 2023Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learning
  8. 2026CDCP: Conditional Diffusion Model with Contextual Prompts for Multi-task Offline Safe Reinforcement Learning