mlsanityData to 14 Oct 2026

DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality

arXiv 2210.13702 ↗First posted 25 Oct 2022cs.RO cs.LGcited by 149

Not on the map. Why?

Cited by 38 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent work has demonstrated the ability of deep reinforcement learning (RL) algorithms to learn complex robotic behaviours in simulation, including in the domain of multi-fingered manipulation. However, such models can be challenging to transfer to the real world due to the gap between simulation and reality. In this paper, we present our techniques to train a) a policy that can perform robust dexterous manipulation on an anthropomorphic robot hand and b) a robust pose estimator suitable for providing reliable real-time information on the state of the object being manipulated. Our policies are trained to adapt to a wide range of conditions in simulation. Consequently, our vision-based policies significantly outperform the best vision policies in the literature on the same reorientation task and are competitive with policies that are given privileged state information via motion capture systems. Our work reaffirms the possibilities of sim-to-real transfer for dexterous manipulation in diverse kinds of hardware and simulator setups, and in our case, with the Allegro Hand and Isaac Gym GPU-based simulation. Furthermore, it opens up possibilities for researchers to achieve such results with commonly-available, affordable robot hands and cameras. Videos of the resulting policy and supplementary information, including experiments and demos, can be found at https://dextreme.org/

Key facts. Cited by 149 papers in the map (data through 14 Oct 2026). Builds on 29 of its 48 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

29 of 48 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2017Mask R-CNNcited by 7,277
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  8. 2018Model Cards for Model Reportingcited by 1,080
  9. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  11. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  12. 2019Solving Rubik's Cube with a Robot Handcited by 734
17 more
  1. 2018Learning Dexterous In-Hand Manipulationcited by 687
  2. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  4. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  5. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  6. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 241
  7. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  8. 2021A System for General In-Hand Object Re-Orientationcited by 165
  9. 2022MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlcited by 80
  10. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  11. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 50
  12. 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 38
  13. 2016Learning Dexterous Manipulation Policies from Experience and Imitationcited by 33
  14. 2022Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planningcited by 26
  15. 2022Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Handcited by 20
  16. 2020Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbscited by 16
  17. 2020Predicting Sim-to-Real Transfer with Probabilistic Dynamics Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 32; 2024: 42; 2025: 40; 2026: 33.

149 papers.

Sort:

By area:

  1. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  2. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
  3. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  4. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  5. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  6. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 169
  7. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  8. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 148
  9. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  10. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 130
  11. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  12. 2022Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapescited by 101
38 more
  1. 2023General In-Hand Object Rotation with Vision and Touchcited by 99
  2. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
  3. 2023Rotating without Seeing: Towards In-hand Dexterity through Touchcited by 84
  4. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 83
  5. 2023LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learningcited by 79
  6. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  7. 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 69
  8. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 66
  9. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  10. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 62
  11. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  12. 2023Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensingcited by 56
  13. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
  14. 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMscited by 52
  15. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 49
  16. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 48
  17. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 46
  18. 2024Twisting Lids Off with Two Handscited by 42
  19. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 41
  20. 2023Dexterous Functional Graspingcited by 39
  21. 2024Hand-Object Interaction Pretraining from Videoscited by 39
  22. 2025DexMachina: Functional Retargeting for Bimanual Dexterous Manipulationcited by 37
  23. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 35
  24. 2023See to Touch: Learning Tactile Dexterity through Visual Incentivescited by 35
  25. 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 31
  26. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 30
  27. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 29
  28. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 28
  29. 2023Neural feels with neural fields: Visuo-tactile perception for in-hand manipulationcited by 28
  30. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 28
  31. 2024ASID: Active Exploration for System Identification in Robotic Manipulationcited by 28
  32. 2024Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewardscited by 28
  33. 2026World-Gymnast: Training Robots with Reinforcement Learning in a World Modelcited by 27
  34. 2023On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfercited by 26
  35. 2024Lessons from Learning to Spin "Pens"cited by 26
  36. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  37. 2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 22
  38. 2024AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touchcited by 21

50 most cited of 149 All 149 (JSON)