mlsanityData to 14 Oct 2026

SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution

arXiv 2312.11598 ↗First posted 18 Dec 2023cs.RO cs.CV cs.LGcited by 70

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Diffusion models have demonstrated strong potential for robotic trajectory planning. However, generating coherent trajectories from high-level instructions remains challenging, especially for long-range composition tasks requiring multiple sequential skills. We propose SkillDiffuser, an end-to-end hierarchical planning framework integrating interpretable skill learning with conditional diffusion planning to address this problem. At the higher level, the skill abstraction module learns discrete, human-understandable skill representations from visual observations and language instructions. These learned skill embeddings are then used to condition the diffusion model to generate customized latent trajectories aligned with the skills. This allows generating diverse state trajectories that adhere to the learnable skills. By integrating skill learning with conditional trajectory generation, SkillDiffuser produces coherent behavior following abstract instructions across diverse tasks. Experiments on multi-task robotic manipulation benchmarks like Meta-World and LOReL demonstrate state-of-the-art performance and human-interpretable skill representations from SkillDiffuser. More visualization results and information could be found on our website.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 40 of its 56 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

40 of 56 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models3 Aug
  2. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL31 May
  3. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning29 May
  4. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learning22 May
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  6. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners3 Feb
  7. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov

Other references · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2016Generative Adversarial Imitation Learningcited by 1,597
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  11. 2018Group Normalizationcited by 1,524
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
21 more
  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  2. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  3. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  4. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  5. 2018Behavioral Cloning from Observationcited by 492
  6. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  7. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  8. 2018Visual Reinforcement Learning with Imagined Goalscited by 251
  9. 2017One-Shot Imitation Learningcited by 233
  10. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  11. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
  12. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  13. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  14. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  15. 2022Causal Transformer for Estimating Counterfactual Outcomescited by 123
  16. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  17. 2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 76
  18. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 58
  19. 2019Sub-policy Adaptation for Hierarchical Reinforcement Learningcited by 47
  20. 2022LISA: Learning Interpretable Skill Abstractions from Languagecited by 30
  21. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 22; 2025: 26; 2026: 22.

70 papers.

Sort:

By area:

  1. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  2. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
  3. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  4. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 74
  5. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 69
  6. 2024Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videoscited by 69
  7. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 69
  8. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 57
  9. 2024MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Controlcited by 53
  10. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  11. 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routingcited by 46
  12. 2024Diffusion Transformer Policycited by 41
38 more
  1. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 41
  2. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 35
  3. 2024G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulationcited by 35
  4. 2024CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Predictioncited by 27
  5. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
  6. 2024Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policiescited by 18
  7. 2024Generative Image as Action Modelscited by 18
  8. 2024Diffusion Model for Planning: A Systematic Literature Reviewcited by 17
  9. 2024AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulationcited by 16
  10. 2024DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulationcited by 15
  11. 2024Language-Guided Object-Centric Diffusion Policy for Generalizable and Collision-Aware Robotic Manipulationcited by 14
  12. 2025Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Frameworkcited by 12
  13. 2025Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learningcited by 12
  14. 2024ComDrive: Comfort-Oriented End-to-End Autonomous Drivingcited by 11
  15. 2025NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistantscited by 11
  16. 2024Forward KL Regularized Preference Optimization for Aligning Diffusion Policiescited by 10
  17. 2025MoE-DP: An MoE-Enhanced Diffusion Policy for Robust Long-Horizon Robotic Manipulation with Skill Decomposition and Failure Recoverycited by 10
  18. 2024A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Modelscited by 8
  19. 2024VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actionscited by 8
  20. 2025Benchmarking Generalizable Bimanual Manipulation: RoboTwin Dual-Arm Collaboration Challenge at CVPR 2025 MEIS Workshopcited by 8
  21. 2025DRAE: Dynamic Retrieval-Augmented Expert Networks for Lifelong Learning and Task Adaptation in Roboticscited by 8
  22. 2025Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policycited by 8
  23. 2025VLN-Zero: Rapid Exploration and Cache-Enabled Neurosymbolic Vision-Language Planning for Zero-Shot Transfer in Robot Navigationcited by 8
  24. 2026Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policiescited by 8
  25. 2024WildLMa: Long Horizon Loco-Manipulation in the Wildcited by 6
  26. 2025CHD: Coupled Hierarchical Diffusion for Long-Horizon Taskscited by 6
  27. 2026Hierarchical Latent Action Modelcited by 6
  28. 2025VER: Vision Expert Transformer for Robot Learning via Foundation Distillation and Dynamic Routingcited by 5
  29. 2024WARPD: World model Assisted Reactive Policy Diffusioncited by 4
  30. 2026Learning Diffusion Policy from Primitive Skills for Robot Manipulationcited by 4
  31. 2025Fast Visuomotor Policy for Robotic Manipulationcited by 3
  32. 2026HeRO: Hierarchical 3D Semantic Representation for Pose-aware Object Manipulationcited by 3
  33. 2026Planning-aligned Token Compression for Long-Context Autonomous Drivingcited by 3
  34. 2026Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learningcited by 3
  35. 2024DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluationcited by 2
  36. 2025HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agentscited by 2
  37. 2025Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Developmentcited by 2
  38. 2026Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigationcited by 2

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulation
  2. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models
  3. 2026Skill-Aware Diffusion for Generalizable Robotic Manipulation
  4. 2025EL3DD: Extended Latent 3D Diffusion for Language Conditioned Multitask Manipulation
  5. 2026DROM: A Language-Guided Diffusion Framework for Multi-Skill Robotic Manipulation
  6. 2025S2-Diffusion: Generalizing from Instance-level to Category-level Skills in Robot Manipulation
  7. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
  8. 2025From Code to Action: Hierarchical Learning of Diffusion-VLM Policies