# Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation

arXiv: https://arxiv.org/abs/2102.11051 · HTML: https://mlsanity.com/paper/2102.11051/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 22 Feb 2021.

## Builds on (most cited first, 22 of 22)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [Go-Explore: a New Approach for Hard-Exploration Problems](https://mlsanity.com/paper/1901.10995/) (2019)
- [A survey on intrinsic motivation in reinforcement learning](https://mlsanity.com/paper/1908.06976/) (2019)
- [Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots](https://mlsanity.com/paper/1301.4862/) (2013)
- [Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning](https://mlsanity.com/paper/1903.08542/) (2019)
- [Energy-Based Hindsight Experience Prioritization](https://mlsanity.com/paper/1810.01363/) (2018)
- [Leveraging Contact Forces for Learning to Grasp](https://mlsanity.com/paper/1809.07004/) (2018)
- [MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning](https://mlsanity.com/paper/1909.04787/) (2019)
- [Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning](https://mlsanity.com/paper/2002.05954/) (2020)
- [Exploring Exploration: Comparing Children with RL Agents in Unified Environments](https://mlsanity.com/paper/2005.02880/) (2020)
- [How Much Do Unstated Problem Constraints Limit Deep Robotic Reinforcement Learning?](https://mlsanity.com/paper/1909.09282/) (2019)

## Built on by (most cited first, 10 of 10)

- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [Enhancing Tactile-based Reinforcement Learning for Robotic Control](https://mlsanity.com/paper/2510.21609/) (2025)
- [TwinTac: A Wide-Range, Highly Sensitive Tactile Sensor with Real-to-Sim Digital Twin Sensor Model](https://mlsanity.com/paper/2509.10063/) (2025)
- [SFP: State-free Priors for Exploration in Off-Policy Reinforcement Learning](https://mlsanity.com/paper/2205.13528/) (2022)
- [Contact Energy Based Hindsight Experience Prioritization](https://mlsanity.com/paper/2312.02677/) (2023)
- [Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing](https://mlsanity.com/paper/2502.19340/) (2025)
- [The Role of Tactile Sensing for Learning Reach and Grasp](https://mlsanity.com/paper/2502.20367/) (2025)
- [The Role of Touch: Towards Optimal Tactile Sensing Distribution in Anthropomorphic Hands for Dexterous In-Hand Manipulation](https://mlsanity.com/paper/2509.14984/) (2025)
- [Zero-shot Whole-Body Manipulation with a Large-Scale Soft Robotic Torso via Guided Reinforcement Learning](https://mlsanity.com/paper/2509.23556/) (2025)
- [Tactile Curiosity Drives Robot Interaction](https://mlsanity.com/paper/2609.40134/) (2026)

## Abstract (arXiv)

In this paper we address the challenge of exploration in deep reinforcement learning for robotic manipulation tasks. In sparse goal settings, an agent does not receive any positive feedback until randomly achieving the goal, which becomes infeasible for longer control sequences. Inspired by touch-based exploration observed in children, we formulate an intrinsic reward based on the sum of forces between a robot's force sensors and manipulation objects that encourages physical interaction. Furthermore, we introduce contact-prioritized experience replay, a sampling scheme that prioritizes contact rich episodes and transitions. We show that our solution accelerates the exploration and outperforms state-of-the-art methods on three fundamental robot manipulation benchmarks.