mlsanityData to 14 Oct 2026

Latent Adaptive Planner for Dynamic Manipulation

arXiv 2505.03077 ↗First posted 6 May 2025cs.RO cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We present the Latent Adaptive Planner (LAP), a trajectory-level latent-variable policy for dynamic nonprehensile manipulation (e.g., box catching) that formulates planning as inference in a low-dimensional latent space and is learned effectively from human demonstration videos. During execution, LAP achieves real-time adaptation by maintaining a posterior over the latent plan and performing variational replanning as new observations arrive. To bridge the embodiment gap between humans and robots, we introduce a model-based proportional mapping that regenerates accurate kinematic-dynamic joint states and object positions from human demonstrations. Through challenging box catching experiments with varying object properties, LAP demonstrates superior success rates, trajectory smoothness, and energy efficiency by learning human-like compliant motions and adaptive behaviors. Overall, LAP enables dynamic manipulation with real-time adaptation and successfully transfer across heterogeneous robot platforms using the same human demonstration videos.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

42 of 60 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  4. 2016Variational Inference: A Review for Statisticianscited by 1,472
  5. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  6. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  9. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  10. 2019Learning Latent Plans from Playcited by 316
  11. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  12. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
30 more
  1. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  2. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  3. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  4. 2020Visual Imitation Made Easycited by 111
  5. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  6. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  7. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 89
  8. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 86
  9. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 82
  10. 2021FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfoldingcited by 81
  11. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  12. 2025Phantom: Training Robots Without Robots Using Only Human Videoscited by 70
  13. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  14. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 63
  15. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 54
  16. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  17. 2024RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learningcited by 46
  18. 2019Learning Predictive Models From Observation and Interactioncited by 41
  19. 2020Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusioncited by 39
  20. 2022Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objectscited by 37
  21. 2023Teleoperation of Humanoid Robots: A Surveycited by 34
  22. 2016More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushingcited by 31
  23. 2025Latent Thought Models with Variational Bayes Inference-Time Computationcited by 25
  24. 2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inferencecited by 14
  25. 2023Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shiftingcited by 8
  26. 2023Molecule Design by Latent Prompt Transformercited by 8
  27. 2023Legged Robots for Object Manipulation: A Reviewcited by 6
  28. 2023Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inferencecited by 6
  29. 2024Latent Space Energy-based Neural ODEscited by 4
  30. 2024Impact-Aware Bimanual Catching of Large-Momentum Objectscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planningcited by 1
  2. 2026AquiLLM: An Architecture for Supporting Tacit Knowledge Capture in Research Groupscited by 1
  3. 2026PhysMani: Physics-principled 3D World Model for Dynamic Object Manipulationcited by 0
  4. 2026Learning to Plan from Random Explorationcited by 0