# Learning Multi-Stage Tasks with One Demonstration via Self-Replay

arXiv: https://arxiv.org/abs/2111.07447 · HTML: https://mlsanity.com/paper/2111.07447/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 14 Nov 2021.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 27 of 27)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [Learning Latent Plans from Play](https://mlsanity.com/paper/1903.01973/) (2019)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [Learning by Playing - Solving Sparse Reward Tasks from Scratch](https://mlsanity.com/paper/1802.10567/) (2018)
- [Deep Spatial Autoencoders for Visuomotor Learning](https://mlsanity.com/paper/1509.06113/) (2015)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [Self-Supervised Correspondence in Visuomotor Policy Learning](https://mlsanity.com/paper/1909.06933/) (2019)
- [Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration](https://mlsanity.com/paper/1707.02920/) (2017)
- [Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration](https://mlsanity.com/paper/2105.06411/) (2021)
- [Neural Task Programming: Learning to Generalize Across Hierarchical Tasks](https://mlsanity.com/paper/1710.01813/) (2017)
- [Task-Embedded Control Networks for Few-Shot Imitation Learning](https://mlsanity.com/paper/1810.03237/) (2018)
- [One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks](https://mlsanity.com/paper/1810.11043/) (2018)
- [FlowControl: Optical Flow Based Visual Servoing](https://mlsanity.com/paper/2007.00291/) (2020)
- [Modeling Long-horizon Tasks as Sequential Interaction Landscapes](https://mlsanity.com/paper/2006.04843/) (2020)
- [Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning](https://mlsanity.com/paper/1908.06769/) (2019)
- [SAFARI: Safe and Active Robot Imitation Learning with Imagination](https://mlsanity.com/paper/2011.09586/) (2020)

## Built on by (most cited first, 21 of 21)

- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation](https://mlsanity.com/paper/2410.11792/) (2024)
- [HYDRA: Hybrid Robot Actions for Imitation Learning](https://mlsanity.com/paper/2306.17237/) (2023)
- [Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction](https://mlsanity.com/paper/2409.18121/) (2024)
- [SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment](https://mlsanity.com/paper/2410.18907/) (2024)
- [Vision-based Manipulation from Single Human Video with Open-World Object Graphs](https://mlsanity.com/paper/2405.20321/) (2024)
- [DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Models](https://mlsanity.com/paper/2402.13181/) (2024)
- [Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning](https://mlsanity.com/paper/2204.02863/) (2022)
- [One-Shot Imitation Learning: A Pose Estimation Perspective](https://mlsanity.com/paper/2310.12077/) (2023)
- [On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation](https://mlsanity.com/paper/2312.12345/) (2023)
- [Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning](https://mlsanity.com/paper/2503.01837/) (2025)
- [DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation](https://mlsanity.com/paper/2510.11258/) (2025)
- [MILES: Making Imitation Learning Easy with Self-Supervision](https://mlsanity.com/paper/2410.19693/) (2024)
- [Subtask-Aware Visual Reward Learning from Segmented Demonstrations](https://mlsanity.com/paper/2502.20630/) (2025)
- [NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors](https://mlsanity.com/paper/2311.01530/) (2023)
- [Back to Reality for Imitation Learning](https://mlsanity.com/paper/2111.12867/) (2021)
- [Conditional Visual Servoing for Multi-Step Tasks](https://mlsanity.com/paper/2205.08441/) (2022)
- [HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learning](https://mlsanity.com/paper/2503.13171/) (2025)
- [All You Need is LUV: Unsupervised Collection of Labeled Images using Invisible UV Fluorescent Indicators](https://mlsanity.com/paper/2203.04566/) (2022)
- [Foundation Models for Rapid Autonomy Validation](https://mlsanity.com/paper/2411.03328/) (2024)
- [Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Experts](https://mlsanity.com/paper/2603.12769/) (2026)

## Abstract (arXiv)

In this work, we introduce a novel method to learn everyday-like multi-stage tasks from a single human demonstration, without requiring any prior object knowledge. Inspired by the recent Coarse-to-Fine Imitation Learning method, we model imitation learning as a learned object reaching phase followed by an open-loop replay of the demonstrator's actions. We build upon this for multi-stage tasks where, following the human demonstration, the robot can autonomously collect image data for the entire multi-stage task, by reaching the next object in the sequence and then replaying the demonstration, and then repeating in a loop for all stages of the task. We evaluate with real-world experiments on a set of everyday-like multi-stage tasks, which we show that our method can solve from a single demonstration. Videos and supplementary material can be found at https://www.robot-learning.uk/self-replay.