# Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost

arXiv: https://arxiv.org/abs/1810.06045 · HTML: https://mlsanity.com/paper/1810.06045/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2018.

## Built on by (most cited first, 30 of 84)

- [Diffusion Policy Policy Optimization](https://mlsanity.com/paper/2409.00588/) (2024)
- [Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning](https://mlsanity.com/paper/2303.05479/) (2023)
- [Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble](https://mlsanity.com/paper/2107.00591/) (2021)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy](https://mlsanity.com/paper/2502.05450/) (2025)
- [Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation](https://mlsanity.com/paper/2403.03949/) (2024)
- [SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2401.16013/) (2024)
- [Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning](https://mlsanity.com/paper/2206.08686/) (2022)
- [GRAPE: Generalizing Robot Policy via Preference Alignment](https://mlsanity.com/paper/2411.19309/) (2024)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention](https://mlsanity.com/paper/2104.11203/) (2021)
- [DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video](https://mlsanity.com/paper/2202.00164/) (2022)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects](https://mlsanity.com/paper/2305.05706/) (2023)
- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [Learning Generalizable Dexterous Manipulation from Human Grasp Affordance](https://mlsanity.com/paper/2204.02320/) (2022)
- [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021)
- [Adaptive Policy Learning for Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2303.07693/) (2023)
- [Dynamic Handover: Throw and Catch with Bimanual Hands](https://mlsanity.com/paper/2309.05655/) (2023)
- [Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning](https://mlsanity.com/paper/2303.01488/) (2023)
- [Autonomous Reinforcement Learning via Subgoal Curricula](https://mlsanity.com/paper/2107.12931/) (2021)
- [MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation](https://mlsanity.com/paper/2309.14236/) (2023)
- [ASID: Active Exploration for System Identification in Robotic Manipulation](https://mlsanity.com/paper/2404.12308/) (2024)
- [Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance](https://mlsanity.com/paper/2212.09902/) (2022)
- [Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot](https://mlsanity.com/paper/2302.09772/) (2023)
- [DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics](https://mlsanity.com/paper/2304.03223/) (2023)
- [DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations](https://mlsanity.com/paper/2209.14284/) (2022)
- [When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning](https://mlsanity.com/paper/2210.10765/) (2022)

## Abstract (arXiv)

Dexterous multi-fingered robotic hands can perform a wide range of manipulation skills, making them an appealing component for general-purpose robotic manipulators. However, such hands pose a major challenge for autonomous control, due to the high dimensionality of their configuration space and complex intermittent contact interactions. In this work, we propose deep reinforcement learning (deep RL) as a scalable solution for learning complex, contact rich behaviors with multi-fingered hands. Deep RL provides an end-to-end approach to directly map sensor readings to actions, without the need for task specific models or policy classes. We show that contact-rich manipulation behavior with multi-fingered hands can be learned by directly training with model-free deep RL algorithms in the real world, with minimal additional assumption and without the aid of simulation. We learn a variety of complex behaviors on two different low-cost hardware platforms. We show that each task can be learned entirely from scratch, and further study how the learning process can be further accelerated by using a small number of human demonstrations to bootstrap learning. Our experiments demonstrate that complex multi-fingered manipulation skills can be learned in the real world in about 4-7 hours for most tasks, and that demonstrations can decrease this to 2-3 hours, indicating that direct deep RL training in the real world is a viable and practical alternative to simulation and model-based control. urlhttps://sites.google.com/view/deeprl-handmanipulation