mlsanityData to 7 Oct 2026

Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

arXiv 1709.10087 ↗First posted 28 Sep 2017cs.LG cs.AI cs.ROcited by 685

Not on the map. Why?

Cited by 220 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Dexterous multi-fingered hands are extremely versatile and provide a generic way to perform a multitude of tasks in human-centric environments. However, effectively controlling them remains challenging due to their high dimensionality and large number of potential contacts. Deep reinforcement learning (DRL) provides a model-agnostic approach to control complex dynamical systems, but has not been shown to scale to high-dimensional dexterous manipulation. Furthermore, deployment of DRL on physical systems remains challenging due to sample inefficiency. Consequently, the success of DRL in robotics has thus far been limited to simpler manipulators and tasks. In this work, we show that model-free DRL can effectively scale up to complex manipulation tasks with a high-dimensional 24-DoF hand, and solve them from scratch in simulated experiments. Furthermore, with the use of a small number of human demonstrations, the sample complexity can be significantly reduced, which enables learning with sample sizes equivalent to a few hours of robot experience. The use of demonstrations result in policies that exhibit very natural movements and, surprisingly, are also substantially more robust.

Key facts. Cited by 685 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 4; 2019: 9; 2020: 15; 2021: 87; 2022: 103; 2023: 122; 2024: 122; 2025: 121; 2026: 102.

685 papers.

Sort:

By area:

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  4. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  5. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  6. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  7. 2021Implicit Behavioral Cloningcited by 442
  8. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  9. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  10. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  11. 2024Diffusion Policy Policy Optimizationcited by 255
  12. 2022Masked Visual Pre-training for Motor Controlcited by 246
38 more
  1. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  2. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  3. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  4. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  5. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  6. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  7. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  8. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  9. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  10. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
  11. 2019Goal-conditioned Imitation Learningcited by 159
  12. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
  13. 2021A System for General In-Hand Object Re-Orientationcited by 155
  14. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  15. 2021Offline RL Without Off-Policy Evaluationcited by 150
  16. 2023Getting aligned on representational alignmentcited by 142
  17. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  18. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 136
  19. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 135
  20. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  21. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  22. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
  23. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  24. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  25. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  26. 2020Primal Wasserstein Imitation Learningcited by 120
  27. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
  28. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
  29. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  30. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  31. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learningcited by 112
  32. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  33. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 103
  34. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
  35. 2022Jump-Start Reinforcement Learningcited by 102
  36. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
  37. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 101
  38. 2024FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulationcited by 99

50 most cited of 685 All 685 (JSON)