# One-Shot Imitation Learning: A Pose Estimation Perspective

arXiv: https://arxiv.org/abs/2310.12077 · HTML: https://mlsanity.com/paper/2310.12077/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 30 of 38)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Simple Open-Vocabulary Object Detection with Vision Transformers](https://mlsanity.com/paper/2205.06230/) (2022)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [Scaling Robot Learning with Semantically Imagined Experience](https://mlsanity.com/paper/2302.11550/) (2023)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [Open-World Object Manipulation using Pre-trained Vision-Language Models](https://mlsanity.com/paper/2303.00905/) (2023)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [Reinforcement and Imitation Learning for Diverse Visuomotor Skills](https://mlsanity.com/paper/1802.09564/) (2018)
- [Visual Imitation Made Easy](https://mlsanity.com/paper/2008.04899/) (2020)
- [Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration](https://mlsanity.com/paper/2105.06411/) (2021)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)
- [You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration](https://mlsanity.com/paper/2201.12716/) (2022)
- [Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations](https://mlsanity.com/paper/2303.01497/) (2023)
- [Model-Based Inverse Reinforcement Learning from Visual Demonstrations](https://mlsanity.com/paper/2010.09034/) (2020)
- [Towards More Generalizable One-shot Visual Imitation Learning](https://mlsanity.com/paper/2110.13423/) (2021)
- [Vision-Based Manipulators Need to Also See from Their Hands](https://mlsanity.com/paper/2203.12677/) (2022)
- [Graph-Structured Visual Imitation](https://mlsanity.com/paper/1907.05518/) (2019)

## Built on by (most cited first, 21 of 21)

- [Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning](https://mlsanity.com/paper/2407.01531/) (2024)
- [DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Models](https://mlsanity.com/paper/2402.13181/) (2024)
- [Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration](https://mlsanity.com/paper/2504.12609/) (2025)
- [DITTO: Demonstration Imitation by Trajectory Transformation](https://mlsanity.com/paper/2403.15203/) (2024)
- [X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real](https://mlsanity.com/paper/2505.07096/) (2025)
- [P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies](https://mlsanity.com/paper/2412.06784/) (2024)
- [On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation](https://mlsanity.com/paper/2312.12345/) (2023)
- [MimicFunc: Imitating Tool Manipulation from a Single Human Video via Functional Correspondence](https://mlsanity.com/paper/2508.13534/) (2025)
- [MILES: Making Imitation Learning Easy with Self-Supervision](https://mlsanity.com/paper/2410.19693/) (2024)
- [FUNCTO: Function-Centric One-Shot Imitation Learning for Tool Manipulation](https://mlsanity.com/paper/2502.11744/) (2025)
- [One-Shot Dual-Arm Imitation Learning](https://mlsanity.com/paper/2503.06831/) (2025)
- [X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations](https://mlsanity.com/paper/2511.04671/) (2025)
- [Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions](https://mlsanity.com/paper/2506.21057/) (2025)
- [Learning in ImaginationLand: Omnidirectional Policies through 3D Generative Models (OP-Gen)](https://mlsanity.com/paper/2509.06191/) (2025)
- [DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulation](https://mlsanity.com/paper/2510.05662/) (2025)
- [Annotation-Free One-Shot Imitation Learning for Multi-Step Manipulation Tasks](https://mlsanity.com/paper/2509.24972/) (2025)
- [Imitation Learning-based Direct Visual Servoing using the Large Projection Formulation](https://mlsanity.com/paper/2406.09120/) (2024)
- [TReF-6: Inferring Task-Relevant Frames from a Single Demonstration for One-Shot Skill Generalization](https://mlsanity.com/paper/2509.00310/) (2025)
- [Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Experts](https://mlsanity.com/paper/2603.12769/) (2026)
- [RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes](https://mlsanity.com/paper/2606.24403/) (2026)
- [RA-VLA: Retrieval-Augmented VLA for Test-Time Adaptation](https://mlsanity.com/paper/2608.25585/) (2026)

## Abstract (arXiv)

In this paper, we study imitation learning under the challenging setting of: (1) only a single demonstration, (2) no further data collection, and (3) no prior task or object knowledge. We show how, with these constraints, imitation learning can be formulated as a combination of trajectory transfer and unseen object pose estimation. To explore this idea, we provide an in-depth study on how state-of-the-art unseen object pose estimators perform for one-shot imitation learning on ten real-world tasks, and we take a deep dive into the effects that camera calibration, pose estimation error, and spatial generalisation have on task success rates. For videos, please visit https://www.robot-learning.uk/pose-estimation-perspective.