# Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1903.08542 · HTML: https://mlsanity.com/paper/1903.08542/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2019.

## Built on by (most cited first, 26 of 26)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies](https://mlsanity.com/paper/2111.02552/) (2021)
- [TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI](https://mlsanity.com/paper/2306.06924/) (2023)
- [The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning](https://mlsanity.com/paper/2311.00924/) (2023)
- [Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration](https://mlsanity.com/paper/2109.08603/) (2021)
- [Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation](https://mlsanity.com/paper/2102.11051/) (2021)
- [Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution](https://mlsanity.com/paper/2404.04253/) (2024)
- [Touch-based Curiosity for Sparse-Reward Tasks](https://mlsanity.com/paper/2104.00442/) (2021)
- [DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation](https://mlsanity.com/paper/2509.18830/) (2025)
- [DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation](https://mlsanity.com/paper/2106.05907/) (2021)
- [ZeBRA: Precisely Destroying Neural Networks with Zero-Data Based Repeated Bit Flip Attack](https://mlsanity.com/paper/2111.01080/) (2021)
- [Learning Gentle Grasping Using Vision, Sound, and Touch](https://mlsanity.com/paper/2503.07926/) (2025)
- [FairDICE: Fairness-Driven Offline Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2506.08062/) (2025)
- [Domain Curiosity: Learning Efficient Data Collection Strategies for Domain Adaptation](https://mlsanity.com/paper/2103.07223/) (2021)
- [Fixed β-VAE Encoding for Curious Exploration in Complex 3D Environments](https://mlsanity.com/paper/2105.08568/) (2021)
- [Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning](https://mlsanity.com/paper/2510.25405/) (2025)
- [Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation](https://mlsanity.com/paper/2602.17921/) (2026)
- [Game-theoretic Objective Space Planning](https://mlsanity.com/paper/2209.07758/) (2022)
- [Constrained Reinforcement Learning for Dexterous Manipulation](https://mlsanity.com/paper/2301.09766/) (2023)
- [Curiosity-Driven Reinforcement Learning based Low-Level Flight Control](https://mlsanity.com/paper/2307.15724/) (2023)
- [TIAGo RL: Simulated Reinforcement Learning Environments with Tactile Data for Mobile Robots](https://mlsanity.com/paper/2311.07260/) (2023)
- [ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation](https://mlsanity.com/paper/2603.10971/) (2026)
- [Learning Hybrid-Control Policies for High-Precision In-Contact Manipulation Under Uncertainty](https://mlsanity.com/paper/2604.19677/) (2026)
- [Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2606.26397/) (2026)
- [Tactile Curiosity Drives Robot Interaction](https://mlsanity.com/paper/2609.40134/) (2026)

## Abstract (arXiv)

Robots must know how to be gentle when they need to interact with fragile objects, or when the robot itself is prone to wear and tear. We propose an approach that enables deep reinforcement learning to train policies that are gentle, both during exploration and task execution. In a reward-based learning environment, a natural approach involves augmenting the (task) reward with a penalty for non-gentleness, which can be defined as excessive impact force. However, augmenting with only this penalty impairs learning: policies get stuck in a local optimum which avoids all contact with the environment. Prior research has shown that combining auxiliary tasks or intrinsic rewards can be beneficial for stabilizing and accelerating learning in sparse-reward domains, and indeed we find that introducing a surprise-based intrinsic reward does avoid the no-contact failure case. However, we show that a simple dynamics-based surprise is not as effective as penalty-based surprise. Penalty-based surprise, based on predicting forceful contacts, has a further benefit: it encourages exploration which is contact-rich yet gentle. We demonstrate the effectiveness of the approach using a complex, tendon-powered robot hand with tactile sensors. Videos are available at http://sites.google.com/view/gentlemanipulation.