# Learning One-Shot Imitation from Humans without Humans

arXiv: https://arxiv.org/abs/1911.01103 · HTML: https://mlsanity.com/paper/1911.01103/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 4 Nov 2019.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Built on by (most cited first, 30 of 33)

- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot](https://mlsanity.com/paper/2307.00595/) (2023)
- [Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation](https://mlsanity.com/paper/2106.12534/) (2021)
- [Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos](https://mlsanity.com/paper/2103.16817/) (2021)
- [Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis](https://mlsanity.com/paper/2312.08782/) (2023)
- [Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction](https://mlsanity.com/paper/2409.18121/) (2024)
- [VLM See, Robot Do: Human Demo Video to Robot Action Plan via Vision Language Model](https://mlsanity.com/paper/2410.08792/) (2024)
- [Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms](https://mlsanity.com/paper/2103.03697/) (2021)
- [One-Shot Imitation Learning: A Pose Estimation Perspective](https://mlsanity.com/paper/2310.12077/) (2023)
- [On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning](https://mlsanity.com/paper/2206.03271/) (2022)
- [Learning Reward Functions for Robotic Manipulation by Observing Humans](https://mlsanity.com/paper/2211.09019/) (2022)
- [Learning Video-Conditioned Policies for Unseen Manipulation Tasks](https://mlsanity.com/paper/2305.06289/) (2023)
- [A Survey on Imitation Learning for Contact-Rich Tasks in Robotics](https://mlsanity.com/paper/2506.13498/) (2025)
- [Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation](https://mlsanity.com/paper/2402.07127/) (2024)
- [Aligning Step-by-Step Instructional Diagrams to Video Demonstrations](https://mlsanity.com/paper/2303.13800/) (2023)
- [EC^2: Emergent Communication for Embodied Control](https://mlsanity.com/paper/2304.09448/) (2023)
- [Guide Your Agent with Adaptive Multimodal Rewards](https://mlsanity.com/paper/2309.10790/) (2023)
- [ILBiT: Imitation Learning for Robot Using Position and Torque Information based on Bilateral Control with Transformer](https://mlsanity.com/paper/2401.16653/) (2024)
- [Effective Tuning Strategies for Generalist Robot Manipulation Policies](https://mlsanity.com/paper/2410.01220/) (2024)
- [Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning](https://mlsanity.com/paper/2410.01529/) (2024)
- [See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations](https://mlsanity.com/paper/2512.07582/) (2025)
- [One-shot Visual Imitation via Attributed Waypoints and Demonstration Augmentation](https://mlsanity.com/paper/2302.04856/) (2023)
- [Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning](https://mlsanity.com/paper/2211.15339/) (2022)
- [A Simple Approach to Continual Learning by Transferring Skill Parameters](https://mlsanity.com/paper/2110.10255/) (2021)
- [Stage Conscious Attention Network (SCAN) : A Demonstration-Conditioned Policy for Few-Shot Imitation](https://mlsanity.com/paper/2112.02278/) (2021)
- [Randomized-to-Canonical Model Predictive Control for Real-world Visual Robotic Manipulation](https://mlsanity.com/paper/2207.01840/) (2022)
- [GIFT: Geometry-Induced Functional Transfer for Category-level Object Manipulation](https://mlsanity.com/paper/2503.15371/) (2025)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [Infinite-dimensional Folded-in-time Deep Neural Networks](https://mlsanity.com/paper/2101.02966/) (2021)

## Abstract (arXiv)

Humans can naturally learn to execute a new task by seeing it performed by other individuals once, and then reproduce it in a variety of configurations. Endowing robots with this ability of imitating humans from third person is a very immediate and natural way of teaching new tasks. Only recently, through meta-learning, there have been successful attempts to one-shot imitation learning from humans; however, these approaches require a lot of human resources to collect the data in the real world to train the robot. But is there a way to remove the need for real world human demonstrations during training? We show that with Task-Embedded Control Networks, we can infer control polices by embedding human demonstrations that can condition a control policy and achieve one-shot imitation learning. Importantly, we do not use a real human arm to supply demonstrations during training, but instead leverage domain randomisation in an application that has not been seen before: sim-to-real transfer on humans. Upon evaluating our approach on pushing and placing tasks in both simulation and in the real world, we show that in comparison to a system that was trained on real-world data we are able to achieve similar results by utilising only simulation data.