# SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation

arXiv: https://arxiv.org/abs/2411.00965 · HTML: https://mlsanity.com/paper/2411.00965/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 1 Nov 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Built on by (most cited first, 30 of 38)

- [UniVLA: Learning to Act Anywhere with Task-centric Latent Actions](https://mlsanity.com/paper/2505.06111/) (2025)
- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models](https://mlsanity.com/paper/2506.16211/) (2025)
- [Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations](https://mlsanity.com/paper/2507.00990/) (2025)
- [CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos](https://mlsanity.com/paper/2601.04061/) (2026)
- [Robot Learning from a Physical World Model](https://mlsanity.com/paper/2511.07416/) (2025)
- [HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos](https://mlsanity.com/paper/2605.24934/) (2026)
- [SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation](https://mlsanity.com/paper/2602.16863/) (2026)
- [Object-centric 3D Motion Field for Robot Learning from Human Videos](https://mlsanity.com/paper/2506.04227/) (2025)
- [GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization](https://mlsanity.com/paper/2605.12369/) (2026)
- [GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learning](https://mlsanity.com/paper/2508.11049/) (2025)
- [FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation](https://mlsanity.com/paper/2602.13444/) (2026)
- [OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer](https://mlsanity.com/paper/2603.14401/) (2026)
- [Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration](https://mlsanity.com/paper/2607.00033/) (2026)
- [V-HOP: Visuo-Haptic 6D Object Pose Tracking](https://mlsanity.com/paper/2502.17434/) (2025)
- [PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Tracking](https://mlsanity.com/paper/2504.20359/) (2025)
- [Multimodal Diffusion Forcing for Forceful Manipulation](https://mlsanity.com/paper/2511.04812/) (2025)
- [BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimation](https://mlsanity.com/paper/2504.07955/) (2025)
- [FUNCanon: Learning Pose-Aware Action Primitives via Functional Object Canonicalization for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2509.19102/) (2025)
- [AFFORD2ACT: Affordance-Guided Automatic Keypoint Selection for Generalizable and Lightweight Robotic Manipulation](https://mlsanity.com/paper/2510.01433/) (2025)
- [Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions](https://mlsanity.com/paper/2506.21057/) (2025)
- [DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulation](https://mlsanity.com/paper/2510.05662/) (2025)
- [Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos](https://mlsanity.com/paper/2602.13197/) (2026)
- [Human Centric General Physical Intelligence for Agile Manufacturing Automation](https://mlsanity.com/paper/2508.11960/) (2025)
- [The Embodiment Gap in Robot Foundation Models](https://mlsanity.com/paper/2608.18433/) (2026)
- [RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learning](https://mlsanity.com/paper/2609.03199/) (2026)
- [Imitation Learning with Precisely Labeled Human Demonstrations](https://mlsanity.com/paper/2504.13803/) (2025)
- [ReSeFlow: Rectifying SE(3)-Equivariant Policy Learning Flows](https://mlsanity.com/paper/2509.22695/) (2025)
- [GLUE: Global-Local Unified Encoding for Imitation Learning via Key-Patch Tracking](https://mlsanity.com/paper/2509.23220/) (2025)
- [EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generation](https://mlsanity.com/paper/2604.01421/) (2026)

## Abstract (arXiv)

We introduce SPOT, an object-centric imitation learning framework. The key idea is to capture each task by an object-centric representation, specifically the SE(3) object pose trajectory relative to the target. This approach decouples embodiment actions from sensory inputs, facilitating learning from various demonstration types, including both action-based and action-less human hand demonstrations, as well as cross-embodiment generalization. Additionally, object pose trajectories inherently capture planning constraints from demonstrations without the need for manually-crafted rules. To guide the robot in executing the task, the object trajectory is used to condition a diffusion policy. We systematically evaluate our method on simulation and real-world tasks. In real-world evaluation, using only eight demonstrations shot on an iPhone, our approach completed all tasks while fully complying with task constraints. Project page: https://nvlabs.github.io/object_centric_diffusion