# Learning Contact-Rich Manipulation Skills with Guided Policy Search

arXiv: https://arxiv.org/abs/1501.05611 · HTML: https://mlsanity.com/paper/1501.05611/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2015.

## Built on by (most cited first, 30 of 38)

- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention](https://mlsanity.com/paper/2104.11203/) (2021)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects](https://mlsanity.com/paper/2305.05706/) (2023)
- [Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2110.14074/) (2021)
- [Offline Meta-Reinforcement Learning for Industrial Insertion](https://mlsanity.com/paper/2110.04276/) (2021)
- [Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks](https://mlsanity.com/paper/2106.14070/) (2021)
- [Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations](https://mlsanity.com/paper/2203.02107/) (2022)
- [Query-Policy Misalignment in Preference-Based Reinforcement Learning](https://mlsanity.com/paper/2305.17400/) (2023)
- [VLANeXt: Recipes for Building Strong VLA Models](https://mlsanity.com/paper/2602.18532/) (2026)
- [Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertainty](https://mlsanity.com/paper/2106.04306/) (2021)
- [Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Tasks](https://mlsanity.com/paper/2007.11646/) (2020)
- [Security Considerations in AI-Robotics: A Survey of Current Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2310.08565/) (2023)
- [DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control](https://mlsanity.com/paper/2306.09010/) (2023)
- [Sample-efficient Adversarial Imitation Learning from Observation](https://mlsanity.com/paper/1906.07374/) (2019)
- [Contact Skill Imitation Learning for Robot-Independent Assembly Programming](https://mlsanity.com/paper/1908.06272/) (2019)
- [Reinforcement Learning using Guided Observability](https://mlsanity.com/paper/2104.10986/) (2021)
- [Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertion](https://mlsanity.com/paper/2203.05955/) (2022)
- [From Noise to Intent: Anchoring Generative VLA Policies with Residual Bridges](https://mlsanity.com/paper/2604.21391/) (2026)
- [Adaptive Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/2105.04699/) (2021)
- [When to Trust Your Data: Enhancing Dyna-Style Model-Based Reinforcement Learning With Data Filter](https://mlsanity.com/paper/2410.12160/) (2024)
- [Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Models](https://mlsanity.com/paper/2503.04280/) (2025)
- [Self-Evaluation in One-Shot Learning from Demonstration of Contact-Intensive Tasks](https://mlsanity.com/paper/1904.01846/) (2019)
- [Learning Reactive and Predictive Differentiable Controllers for Switching Linear Dynamical Models](https://mlsanity.com/paper/2103.14256/) (2021)
- [Explaining the Decisions of Deep Policy Networks for Robotic Manipulations](https://mlsanity.com/paper/2310.19432/) (2023)
- [Dual-Arm Adversarial Robot Learning](https://mlsanity.com/paper/2110.08066/) (2021)
- [Space Non-cooperative Object Active Tracking with Deep Reinforcement Learning](https://mlsanity.com/paper/2112.09854/) (2021)
- [Reinforcement Learning from Demonstrations by Novel Interactive Expert and Application to Automatic Berthing Control Systems for Unmanned Surface Vessel](https://mlsanity.com/paper/2202.11325/) (2022)

## Abstract (arXiv)

Autonomous learning of object manipulation skills can enable robots to acquire rich behavioral repertoires that scale to the variety of objects found in the real world. However, current motion skill learning methods typically restrict the behavior to a compact, low-dimensional representation, limiting its expressiveness and generality. In this paper, we extend a recently developed policy search method citela-lnnpg-14 and use it to learn a range of dynamic manipulation behaviors with highly general policy representations, without using known models or example demonstrations. Our approach learns a set of trajectories for the desired motion skill by using iteratively refitted time-varying linear models, and then unifies these trajectories into a single control policy that can generalize to new situations. To enable this method to run on a real robot, we introduce several improvements that reduce the sample count and automate parameter selection. We show that our method can acquire fast, fluent behaviors after only minutes of interaction time, and can learn robust controllers for complex tasks, including putting together a toy airplane, stacking tight-fitting lego blocks, placing wooden rings onto tight-fitting pegs, inserting a shoe tree into a shoe, and screwing bottle caps onto bottles.