mlsanityData to 7 Oct 2026

SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation

arXiv 2411.00965 ↗First posted 1 Nov 2024cs.ROcited by 36

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

We introduce SPOT, an object-centric imitation learning framework. The key idea is to capture each task by an object-centric representation, specifically the SE(3) object pose trajectory relative to the target. This approach decouples embodiment actions from sensory inputs, facilitating learning from various demonstration types, including both action-based and action-less human hand demonstrations, as well as cross-embodiment generalization. Additionally, object pose trajectories inherently capture planning constraints from demonstrations without the need for manually-crafted rules. To guide the robot in executing the task, the object trajectory is used to condition a diffusion policy. We systematically evaluate our method on simulation and real-world tasks. In real-world evaluation, using only eight demonstrations shot on an iPhone, our approach completed all tasks while fully complying with task constraints. Project page: https://nvlabs.github.io/object_centric_diffusion

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 1 Nov 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 19; 2026: 17.

36 papers.

Sort:

By area:

  1. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  2. 2025Humanoid Policy ~ Human Policycited by 65
  3. 2025ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Modelscited by 43
  4. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  5. 2026CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videoscited by 27
  6. 2025Robot Learning from a Physical World Modelcited by 18
  7. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 14
  8. 2025Object-centric 3D Motion Field for Robot Learning from Human Videoscited by 13
  9. 2026SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulationcited by 10
  10. 2025GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learningcited by 8
  11. 2026GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specializationcited by 7
  12. 2026FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulationcited by 6
24 more
  1. 2026Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstrationcited by 6
  2. 2025PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Trackingcited by 5
  3. 2026OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfercited by 5
  4. 2025V-HOP: Visuo-Haptic 6D Object Pose Trackingcited by 4
  5. 2025BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimationcited by 3
  6. 2025FUNCanon: Learning Pose-Aware Action Primitives via Functional Object Canonicalization for Generalizable Robotic Manipulationcited by 3
  7. 2025AFFORD2ACT: Affordance-Guided Automatic Keypoint Selection for Generalizable and Lightweight Robotic Manipulationcited by 3
  8. 2025Multimodal Diffusion Forcing for Forceful Manipulationcited by 3
  9. 2025Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditionscited by 2
  10. 2025DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulationcited by 2
  11. 2025Human Centric General Physical Intelligence for Agile Manufacturing Automationcited by 1
  12. 2026Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videoscited by 1
  13. 2025Imitation Learning with Precisely Labeled Human Demonstrationscited by 0
  14. 2025ReSeFlow: Rectifying SE(3)-Equivariant Policy Learning Flowscited by 0
  15. 2025GLUE: Global-Local Unified Encoding for Imitation Learning via Key-Patch Trackingcited by 0
  16. 2026EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generationcited by 0
  17. 2026Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Cameracited by 0
  18. 2026Human2Any: Human-to-Robot Transfer via Constraint-Aware Compositional Planningcited by 0
  19. 2026EmbodimentSemantic: A Spatial Scene-Graph Dataset and Benchmark for Vision-Language Models on Embodied Manipulation Trajectoriescited by 0
  20. 2026The Embodiment Gap in Robot Foundation Modelscited by 0
  21. 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learningcited by 0
  22. 2026GeomVLA: Unifying Scene, Motion, and Action in 3Dcited by 0
  23. 2026From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrationscited by 0
  24. 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Motion Before Action: Diffusing Object Motion as Manipulation Condition
  2. 2026Hand-centric Human-to-Robot Trajectory Transfer from Video Demonstrations via Object Category-agnostic Temporal Localization
  3. 2024Language-Guided Object-Centric Diffusion Policy for Generalizable and Collision-Aware Robotic Manipulation
  4. 2026RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes
  5. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations
  6. 2023One-Shot Imitation Learning: A Pose Estimation Perspective
  7. 2026More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning
  8. 2025DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy