mlsanityData to 14 Oct 2026

Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids

arXiv 2502.20396 ↗First posted 27 Feb 2025cs.RO cs.AI cs.CVcited by 48

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Learning generalizable robot manipulation policies, especially for complex multi-fingered humanoids, remains a significant challenge. Existing approaches primarily rely on extensive data collection and imitation learning, which are expensive, labor-intensive, and difficult to scale. Sim-to-real reinforcement learning (RL) offers a promising alternative, but has mostly succeeded in simpler state-based or single-hand setups. How to effectively extend this to vision-based, contact-rich bimanual manipulation tasks remains an open question. In this paper, we introduce a practical sim-to-real RL recipe that trains a humanoid robot to perform three challenging dexterous manipulation tasks: grasp-and-reach, box lift and bimanual handover. Our method features an automated real-to-sim tuning module, a generalized reward formulation based on contact and object goals, a divide-and-conquer policy distillation framework, and a hybrid object representation strategy with modality-specific augmentation. We demonstrate high success rates on unseen objects and robust, adaptive policy behaviors -- highlighting that vision-based dexterous manipulation via sim-to-real RL is not only viable, but also scalable and broadly applicable to real-world humanoid manipulation tasks.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 56 of its 68 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

56 of 68 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots10 Sep
  2. 2024ASID: Active Exploration for System Identification in Robotic Manipulation18 Apr
  3. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation6 Mar

Other references · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  7. 2018Group Normalizationcited by 1,524
  8. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  11. 2018Exploration by Random Network Distillationcited by 993
  12. 2018DeepMind Control Suitecited by 956
38 more
  1. 2017Deep Reinforcement Learning that Matterscited by 754
  2. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  3. 2019Solving Rubik's Cube with a Robot Handcited by 734
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  7. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  8. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  9. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  10. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  11. 2017Count-Based Exploration with Neural Density Modelscited by 257
  12. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189
  13. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  14. 2021A System for General In-Hand Object Re-Orientationcited by 165
  15. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  16. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  17. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  18. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
  19. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 134
  20. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  21. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  22. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  23. 2023General In-Hand Object Rotation with Vision and Touchcited by 99
  24. 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 93
  25. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  26. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 62
  27. 2024WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contactscited by 61
  28. 2022Advanced Skills by Learning Locomotion and Local Navigation End-to-Endcited by 52
  29. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 49
  30. 2024DextrAH-RGB: Visuomotor Policies to Grasp Anything with Dexterous Handscited by 45
  31. 2024Twisting Lids Off with Two Handscited by 42
  32. 2024DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabricscited by 42
  33. 2024Object-Centric Dexterous Manipulation from Human Motion Datacited by 41
  34. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 35
  35. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 30
  36. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 29
  37. 2025DexterityGen: Foundation Controller for Unprecedented Dexteritycited by 29
  38. 2024Lessons from Learning to Spin "Pens"cited by 26

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 28; 2026: 20.

48 papers.

Sort:

By area:

  1. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 59
  2. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 35
  3. 2025Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactionscited by 30
  4. 2025EmbodieDreamer: Advancing Real2Sim2Real Transfer for Policy Training via Embodied World Modelingcited by 29
  5. 2025HuB: Learning Extreme Humanoid Balancecited by 28
  6. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  7. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
  8. 2025SIMS-V: Simulated Instruction-Tuning for Spatial Video Understandingcited by 16
  9. 2025What Matters for Batch Online Reinforcement Learning in Robotics?cited by 15
  10. 2025DreamControl: Human-Inspired Whole-Body Humanoid Control for Scene Interaction via Guided Diffusioncited by 15
  11. 2026Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulationcited by 14
  12. 2026Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrationscited by 12
36 more
  1. 2026ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Controlcited by 11
  2. 2025End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by Autonomous Hand VLA Policy for Efficient Data Collectioncited by 9
  3. 2025Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robotscited by 9
  4. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 9
  5. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  6. 2025Towards Adaptable Humanoid Control via Adaptive Motion Trackingcited by 8
  7. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoidscited by 7
  8. 2025Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulationcited by 5
  9. 2025Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learningcited by 5
  10. 2025EMAC+: Embodied Multimodal Agent for Collaborative Planning with VLM+LLMcited by 5
  11. 2025End-to-end RL Improves Dexterous Grasping Policiescited by 5
  12. 2025Universal Dexterous Functional Grasping via Demonstration-Editing Reinforcement Learningcited by 4
  13. 2026A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulationcited by 4
  14. 2025Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learningcited by 3
  15. 2026DeFM: Learning Foundation Representations from Depth for Roboticscited by 3
  16. 2025Learning Predictive Visuomotor Coordinationcited by 2
  17. 2025Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulationcited by 2
  18. 2026Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulationcited by 2
  19. 2026Tether: Autonomous Functional Play with Correspondence-Driven Trajectory Warpingcited by 2
  20. 2026CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Drivingcited by 2
  21. 2025Efficient Model-Based Reinforcement Learning for Robot Control via Online Optimizationcited by 1
  22. 2025Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learningcited by 1
  23. 2026Data-Efficient Hierarchical Goal-Conditioned Reinforcement Learning via Normalizing Flowscited by 1
  24. 2026How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preferencecited by 1
  25. 2025Learning Dexterous Object Handovercited by 0
  26. 2025Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrationscited by 0
  27. 2025Learning to Grasp Anything by Playing with Random Toyscited by 0
  28. 2026HierKick: Hierarchical Reinforcement Learning for Vision-Guided Soccer Robot Controlcited by 0
  29. 2026ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulationcited by 0
  30. 2026Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representationcited by 0
  31. 2026Shield-Loco: Shielding Locomotion Policies with Predictive Safety Filteringcited by 0
  32. 2026Mana: Dexterous Manipulation of Articulated Toolscited by 0
  33. 2026Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulationcited by 0
  34. 2026SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routingcited by 0
  35. 2026MASkillBlender: Decentralized Whole-Body Coordination for Multi-Humanoid Loco-Manipulation via Skill Blendingcited by 0
  36. 2026PatternDex: Learning Interaction Patterns to Guide Reinforcement Learning of Bimanual Dexterous Manipulation of Articulated Objectscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  2. 2026SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
  3. 2025Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey
  4. 2022Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrations
  5. 2024Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper
  6. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
  7. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration
  8. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models