# One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL

arXiv: https://arxiv.org/abs/1810.05017 · HTML: https://mlsanity.com/paper/1810.05017/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2018.

## Built on by (most cited first, 11 of 11)

- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [VIMA: General Robot Manipulation with Multimodal Prompts](https://mlsanity.com/paper/2210.03094/) (2022)
- [Watch, Try, Learn: Meta-Learning from Demonstrations and Reward](https://mlsanity.com/paper/1906.03352/) (2019)
- [PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning](https://mlsanity.com/paper/2102.12560/) (2021)
- [Scalable Multi-Task Imitation Learning with Autonomous Improvement](https://mlsanity.com/paper/2003.02636/) (2020)
- [Wish you were here: Hindsight Goal Selection for long-horizon dexterous manipulation](https://mlsanity.com/paper/2112.00597/) (2021)
- [Simitate: A Hybrid Imitation Learning Benchmark](https://mlsanity.com/paper/1905.06002/) (2019)
- [EC^2: Emergent Communication for Embodied Control](https://mlsanity.com/paper/2304.09448/) (2023)
- [D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioning](https://mlsanity.com/paper/2210.14428/) (2022)
- [Accelerate Reinforcement Learning with PID Controllers in the Pendulum Simulations](https://mlsanity.com/paper/2210.00770/) (2022)
- [Imitating careful experts to avoid catastrophic events](https://mlsanity.com/paper/2302.01193/) (2023)

## Abstract (arXiv)

Humans are experts at high-fidelity imitation -- closely mimicking a demonstration, often in one attempt. Humans use this ability to quickly solve a task instance, and to bootstrap learning of new tasks. Achieving these abilities in autonomous agents is an open problem. In this paper, we introduce an off-policy RL algorithm (MetaMimic) to narrow this gap. MetaMimic can learn both (i) policies for high-fidelity one-shot imitation of diverse novel skills, and (ii) policies that enable the agent to solve tasks more efficiently than the demonstrators. MetaMimic relies on the principle of storing all experiences in a memory and replaying these to learn massive deep neural network policies by off-policy RL. This paper introduces, to the best of our knowledge, the largest existing neural networks for deep RL and shows that larger networks with normalization are needed to achieve one-shot high-fidelity imitation on a challenging manipulation task. The results also show that both types of policy can be learned from vision, in spite of the task rewards being sparse, and without access to demonstrator actions.