# Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation

arXiv: https://arxiv.org/abs/2008.07682 · HTML: https://mlsanity.com/paper/2008.07682/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 18 Aug 2020.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [Deep Reinforcement Learning for High Precision Assembly Tasks](https://mlsanity.com/paper/1708.04033/) (2017)
- [Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly](https://mlsanity.com/paper/1903.01066/) (2019)
- [Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning](https://mlsanity.com/paper/2005.10872/) (2020)
- [Towards Orientation Learning and Adaptation in Cartesian Space](https://mlsanity.com/paper/1907.03918/) (2019)
- [Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks](https://mlsanity.com/paper/1710.08555/) (2017)

## Built on by (most cited first, 28 of 28)

- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [From Imitation to Refinement -- Residual RL for Precise Assembly](https://mlsanity.com/paper/2407.16677/) (2024)
- [ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning](https://mlsanity.com/paper/2503.21860/) (2025)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [JUICER: Data-Efficient Imitation Learning for Robotic Assembly](https://mlsanity.com/paper/2404.03729/) (2024)
- [DEFT: Dexterous Fine-Tuning for Real-World Hand Policies](https://mlsanity.com/paper/2310.19797/) (2023)
- [A Survey on Imitation Learning for Contact-Rich Tasks in Robotics](https://mlsanity.com/paper/2506.13498/) (2025)
- [Wish you were here: Hindsight Goal Selection for long-horizon dexterous manipulation](https://mlsanity.com/paper/2112.00597/) (2021)
- [Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Grasping](https://mlsanity.com/paper/2410.02475/) (2024)
- [RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulation](https://mlsanity.com/paper/2602.01789/) (2026)
- [GenCHiP: Generating Robot Policy Code for High-Precision and Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2404.06645/) (2024)
- [Few-Shot Learning of Force-Based Motions From Demonstration Through Pre-training of Haptic Representation](https://mlsanity.com/paper/2309.04640/) (2023)
- [Fast Ergodic Search with Kernel Functions](https://mlsanity.com/paper/2403.01536/) (2024)
- [RESPRECT: Speeding-up Multi-fingered Grasping with Residual Reinforcement Learning](https://mlsanity.com/paper/2401.14858/) (2024)
- [Fusion Dynamical Systems with Machine Learning in Imitation Learning: A Comprehensive Overview](https://mlsanity.com/paper/2403.19916/) (2024)
- [Efficient Skill Acquisition for Complex Manipulation Tasks in Obstructed Environments](https://mlsanity.com/paper/2303.03365/) (2023)
- [Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning](https://mlsanity.com/paper/2605.19919/) (2026)
- [CHEQ-ing the Box: Safe Variable Impedance Learning for Robotic Polishing](https://mlsanity.com/paper/2501.07985/) (2025)
- [Dynamic Action Interpolation: A Universal Approach for Accelerating Reinforcement Learning with Expert Guidance](https://mlsanity.com/paper/2504.18766/) (2025)
- [Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulation](https://mlsanity.com/paper/2505.10522/) (2025)
- [MoRe-ERL: Learning Motion Residuals using Episodic Reinforcement Learning](https://mlsanity.com/paper/2508.01409/) (2025)
- [Reinforcement Learning for Robotic Insertion of Flexible Cables in Industrial Settings](https://mlsanity.com/paper/2509.13731/) (2025)
- [VE2VF: Vision-Enabled to Vision-Free Distillation via Real-world Reinforcement Learning for Robust Contact-Rich Manipulation](https://mlsanity.com/paper/2605.29564/) (2026)
- [V2P-Manip: Learning Dexterous Manipulation from Monocular Human Videos](https://mlsanity.com/paper/2606.16436/) (2026)
- [Learning Residual Kinematic Corrections for Continuous Neural Decoding via Reinforcement Learning](https://mlsanity.com/paper/2607.11530/) (2026)
- [Safety-aware Skill Adaptation for Reinforcement Learning in Dynamic Environments](https://mlsanity.com/paper/2609.11433/) (2026)
- [Hybrid Residual Reinforcement Learning for Contact-Rich Robotic Book Insertion](https://mlsanity.com/paper/2609.19962/) (2026)
- [Potential-Field Action Representation for Reinforcement Learning in Contact-Rich Manipulation](https://mlsanity.com/paper/2609.21609/) (2026)

## Abstract (arXiv)

Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can learn such skills. Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC) but can struggle in the context of insertion. Policy adaptation strategies such as residual learning can help improve the overall performance of policies in the context of contact-rich manipulation. However, it is not clear how to best do this with DMPs. As a result, we consider several possible ways for adapting a DMP formulation and propose ``residual Learning from Demonstration`` (rLfD), a framework that combines DMPs with Reinforcement Learning (RL) to learn a residual correction policy. Our evaluations suggest that applying residual learning directly in task space and operating on the full pose of the robot can significantly improve the overall performance of DMPs. We show that rLfD offers a gentle to the joints solution that improves the task success and generalisation of DMPs rband enables transfer to different geometries and frictions through few-shot task adaptation. The proposed framework is evaluated on a set of tasks. A simulated robot and a physical robot have to successfully insert pegs, gears and plugs into their respective sockets. Other material and videos accompanying this paper are provided at https://sites.google.com/view/rlfd/.