# One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors

arXiv: https://arxiv.org/abs/1509.06841 · HTML: https://mlsanity.com/paper/1509.06841/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2015.

## Built on by (most cited first, 26 of 26)

- [Supervised Pretraining Can Learn In-Context Reinforcement Learning](https://mlsanity.com/paper/2306.14892/) (2023)
- [Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation](https://mlsanity.com/paper/2408.11812/) (2024)
- [Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining](https://mlsanity.com/paper/2310.08566/) (2023)
- [Investigating Compounding Prediction Errors in Learned Dynamics Models](https://mlsanity.com/paper/2203.09637/) (2022)
- [A Survey of Embodied Learning for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2408.11537/) (2024)
- [Context is Everything: Implicit Identification for Dynamics Adaptation](https://mlsanity.com/paper/2203.05549/) (2022)
- [Task-Agnostic Morphology Evolution](https://mlsanity.com/paper/2102.13100/) (2021)
- [Training Efficient Controllers via Analytic Policy Gradient](https://mlsanity.com/paper/2209.13052/) (2022)
- [Live in the Moment: Learning Dynamics Model Adapted to Evolving Policy](https://mlsanity.com/paper/2207.12141/) (2022)
- [Query-Policy Misalignment in Preference-Based Reinforcement Learning](https://mlsanity.com/paper/2305.17400/) (2023)
- [Rapid Motor Adaptation for Robotic Manipulator Arms](https://mlsanity.com/paper/2312.04670/) (2023)
- [Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control](https://mlsanity.com/paper/2210.12583/) (2022)
- [Learning to Optimize in Model Predictive Control](https://mlsanity.com/paper/2212.02603/) (2022)
- [Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertainty](https://mlsanity.com/paper/2106.04306/) (2021)
- [LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Tasks](https://mlsanity.com/paper/2107.04775/) (2021)
- [Learning Sampling Distributions for Model Predictive Control](https://mlsanity.com/paper/2212.02587/) (2022)
- [Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning](https://mlsanity.com/paper/2406.05064/) (2024)
- [SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors](https://mlsanity.com/paper/2606.23444/) (2026)
- [Deep Model Predictive Optimization](https://mlsanity.com/paper/2310.04590/) (2023)
- [Self-Improving Semantic Perception for Indoor Localisation](https://mlsanity.com/paper/2105.01595/) (2021)
- [Ensuring Force Safety in Vision-Guided Robotic Manipulation via Implicit Tactile Calibration](https://mlsanity.com/paper/2412.10349/) (2024)
- [Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation](https://mlsanity.com/paper/2306.13554/) (2023)
- [Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning](https://mlsanity.com/paper/2309.05582/) (2023)
- [Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions](https://mlsanity.com/paper/2405.16184/) (2024)
- [Fully Distributed Actor-Critic Architecture for Multitask Deep Reinforcement Learning](https://mlsanity.com/paper/2110.12306/) (2021)
- [Transfer Learning for Control Systems via Neural Simulation Relations](https://mlsanity.com/paper/2412.01783/) (2024)

## Abstract (arXiv)

One of the key challenges in applying reinforcement learning to complex robotic control tasks is the need to gather large amounts of experience in order to find an effective policy for the task at hand. Model-based reinforcement learning can achieve good sample efficiency, but requires the ability to learn a model of the dynamics that is good enough to learn an effective policy. In this work, we develop a model-based reinforcement learning algorithm that combines prior knowledge from previous tasks with online adaptation of the dynamics model. These two ingredients enable highly sample-efficient learning even in regimes where estimating the true dynamics is very difficult, since the online model adaptation allows the method to locally compensate for unmodeled variation in the dynamics. We encode the prior experience into a neural network dynamics model, adapt it online by progressively refitting a local linear model of the dynamics, and use model predictive control to plan under these dynamics. Our experimental results show that this approach can be used to solve a variety of complex robotic manipulation tasks in just a single attempt, using prior data from other manipulation behaviors.