mlsanityData to 14 Oct 2026

Conditional Diffusion Model for Multi-Agent Dynamic Task Decomposition

arXiv 2511.13137 ↗First posted 17 Nov 2025cs.AIcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Task decomposition has shown promise in complex cooperative multi-agent reinforcement learning (MARL) tasks, which enables efficient hierarchical learning for long-horizon tasks in dynamic and uncertain environments. However, learning dynamic task decomposition from scratch generally requires a large number of training samples, especially exploring the large joint action space under partial observability. In this paper, we present the Conditional Diffusion Model for Dynamic Task Decomposition (CD3T), a novel two-level hierarchical MARL framework designed to automatically infer subtask and coordination patterns. The high-level policy learns subtask representation to generate a subtask selection strategy based on subtask effects. To capture the effects of subtasks on the environment, CD3T predicts the next observation and reward using a conditional diffusion model. At the low level, agents collaboratively learn and share specialized skills within their assigned subtasks. Moreover, the learned subtask representation is also used as additional semantic information in a multi-head attention mixing network to enhance value decomposition and provide an efficient reasoning bridge between individual and joint value functions. Experimental results on various benchmarks demonstrate that CD3T achieves better performance than existing baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 29 of its 51 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

29 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning29 May

Other references · 28

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2013Auto-Encoding Variational Bayescited by 4,053
  5. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  6. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  8. 2019The StarCraft Multi-Agent Challengecited by 597
  9. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  10. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  11. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  12. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
16 more
  1. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  2. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  3. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  4. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  5. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 98
  6. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
  7. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 86
  8. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  9. 2017Guided Deep Reinforcement Learning for Swarm Systemscited by 44
  10. 2019Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discoverycited by 43
  11. 2020UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learningcited by 33
  12. 2018Cooperative and Distributed Reinforcement Learning of Drones for Field Coveragecited by 20
  13. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  14. 2017Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learningcited by 15
  15. 2022MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Treescited by 11
  16. 2025High-order Interactions Modeling for Interpretable Multi-Agent Q-Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Symbolic Task Decompositions for Multi-Agent Teams
  2. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards
  3. 2026Adaptive Value Decomposition: Coordinating a Varying Number of Agents in Urban Systems
  4. 2025Multi-Task Multi-Agent Reinforcement Learning via Skill Graphs
  5. 2025CTC: The Composite Task Challenge for Cooperative Multi-Agent Reinforcement Learning
  6. 2026Reinforcement Learning with Decomposed Subtasks
  7. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning
  8. 2025Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning