mlsanityData to 14 Oct 2026

Latent Diffusion Planning for Imitation Learning

arXiv 2504.16925 ↗First posted 23 Apr 2025cs.RO cs.AIcited by 26

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Recent progress in imitation learning has been enabled by policy architectures that scale to complex visuomotor tasks, multimodal distributions, and large datasets. However, these methods often rely on learning from large amount of expert demonstrations. To address these shortcomings, we propose Latent Diffusion Planning (LDP), a modular approach consisting of a planner which can leverage action-free demonstrations, and an inverse dynamics model which can leverage suboptimal data, that both operate over a learned latent space. First, we learn a compact latent space through a variational autoencoder, enabling effective forecasting of future states in image-based domains. Then, we train a planner and an inverse dynamics model with diffusion objectives. By separating planning from action prediction, LDP can benefit from the denser supervision signals of suboptimal and action-free data. On simulated visual robotic manipulation tasks, LDP outperforms state-of-the-art imitation learning approaches, as they cannot leverage such additional data.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 50 of its 59 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

50 of 59 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models16 Oct
  2. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies20 Apr
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  4. 2023Imitating Human Behaviour with Diffusion Models25 Jan
  5. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov

Other references · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  9. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  10. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  12. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
33 more
  1. 2018World Modelscited by 1,513
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  7. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  8. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  9. 2024Genie: Generative Interactive Environmentscited by 747
  10. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  11. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  12. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  13. 2018Behavioral Cloning from Observationcited by 492
  14. 2023Learning Interactive Real-World Simulatorscited by 458
  15. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  16. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  17. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  18. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  19. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  20. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  21. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  22. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 134
  23. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  24. 2020Visual Imitation Made Easycited by 111
  25. 2019Reward-Conditioned Policiescited by 102
  26. 2018Imitating Latent Policies from Observationcited by 101
  27. 2024Video as the New Language for Real-World Decision Makingcited by 87
  28. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 62
  29. 2023Robot Learning with Sensorimotor Pre-trainingcited by 51
  30. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  31. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 44
  32. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 42
  33. 2019Learning Predictive Models From Observation and Interactioncited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 20.

26 papers.

Sort:

By area:

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  2. 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representationscited by 15
  3. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  4. 2026Latent Geometry Beyond Search: Amortizing Planning in World Modelscited by 15
  5. 2026Language-Conditioned World Modeling for Visual Navigationcited by 10
  6. 2025Benchmarking Vision, Language, & Action Models in Procedurally Generated, Open Ended Action Environmentscited by 6
  7. 2025Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impactcited by 6
  8. 2026Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptationcited by 6
  9. 2026One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policycited by 6
  10. 2026FF-JEPA: Long-Horizon Planning in World Models with Latent Plannerscited by 6
  11. 2026ACID: Action Consistency via Inverse Dynamics for Planning with World Modelscited by 5
  12. 2026ODEWorld: A Continuous Predictive Architecture via Physical-Time Flowcited by 4
14 more
  1. 2025Model-Based Diffusion Sampling for Predictive Control in Offline Decision Makingcited by 3
  2. 2026When Does Predictive Inverse Dynamics Outperform Behavior Cloning?cited by 3
  3. 2026Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policiescited by 3
  4. 2026OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulationcited by 3
  5. 2026Structured 4D Latent Predictive Model for Robot Planningcited by 3
  6. 2025Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Developmentcited by 2
  7. 2026Compositional Visual Planning via Inference-Time Diffusion Scalingcited by 2
  8. 2026Refining Compositional Diffusion for Reliable Long-Horizon Planningcited by 2
  9. 2025Retrieve-Augmented Generation for Speeding up Diffusion Policy without Additional Trainingcited by 1
  10. 2026Contractive Diffusion Policies: Robust Action Diffusion via Contractive Score-Based Sampling with Differential Equationscited by 1
  11. 2026Online Self-Training for Co-Adaptation in Hierarchical Diffusion Policiescited by 1
  12. 2026Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulationcited by 1
  13. 2026Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundationscited by 0
  14. 2026Augmentations for Robust and Efficient Imitation Learning in Streamed Video Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Imitation Learning with Limited Actions via Diffusion Planners and Deep Koopman Controllers
  2. 2026Latent Diffusion Policy: Shaping Latent Spaces for Diffusion-Based Robotic Manipulation
  3. 2025Diffusion Models for Robotic Manipulation: A Survey
  4. 2025Demystifying Diffusion Policies: Action Memorization and Simple Lookup Table Alternatives
  5. 2026GraspLDP: Towards Generalizable Grasping Policy via Latent Diffusion
  6. 2024Unpacking the Individual Components of Diffusion Policy
  7. 2025Spatial-Temporal Aware Visuomotor Diffusion Policy Learning
  8. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations