mlsanityData to 7 Oct 2026

Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning

arXiv 2206.08686 ↗First posted 17 Jun 2022cs.RO cs.AI cs.LGcited by 113

Not on the map. Why?

Abstract (arXiv)

Achieving human-level dexterity is an important open problem in robotics. However, tasks of dexterous hand manipulation, even at the baby level, are challenging to solve through reinforcement learning (RL). The difficulty lies in the high degrees of freedom and the required cooperation among heterogeneous agents (e.g., joints of fingers). In this study, we propose the Bimanual Dexterous Hands Benchmark (Bi-DexHands), a simulator that involves two dexterous hands with tens of bimanual manipulation tasks and thousands of target objects. Specifically, tasks in Bi-DexHands are designed to match different levels of human motor skills according to cognitive science literature. We built Bi-DexHands in the Issac Gym; this enables highly efficient RL training, reaching 30,000+ FPS by only one single NVIDIA RTX 3090. We provide a comprehensive benchmark for popular RL algorithms under different settings; this includes Single-agent/Multi-agent RL, Offline RL, Multi-task RL, and Meta RL. Our results show that the PPO type of on-policy algorithms can master simple manipulation tasks that are equivalent up to 48-month human babies (e.g., catching a flying object, opening a bottle), while multi-agent RL can further help to master manipulations that require skilled bimanual cooperation (e.g., lifting a pot, stacking blocks). Despite the success on each single task, when it comes to acquiring multiple manipulation skills, existing RL algorithms fail to work in most of the multi-task and the few-shot learning settings, which calls for more substantial development from the RL community. Our project is open sourced at https://github.com/PKU-MARL/DexterousHands.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 37 of its 71 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

37 of 71 references.

References · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  8. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  11. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  12. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
25 more
  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  2. 2019Solving Rubik's Cube with a Robot Handcited by 699
  3. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  4. 2019The StarCraft Multi-Agent Challengecited by 589
  5. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  7. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  8. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  9. 2022Masked Visual Pre-training for Motor Controlcited by 246
  10. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  11. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  12. 2021A System for General In-Hand Object Re-Orientationcited by 155
  13. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  14. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  15. 2013Data-Driven Grasp Synthesis - A Surveycited by 104
  16. 2020Game-Theoretic Multiagent Reinforcement Learningcited by 92
  17. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  18. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  19. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
  20. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
  21. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  22. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 30
  23. 2021On the Complexity of Computing Markov Perfect Equilibrium in General-Sum Stochastic Gamescited by 18
  24. 2017Reinforcement Learning for Pivoting Taskcited by 17
  25. 2020Solving Challenging Dexterous Manipulation Tasks With Trajectory Optimisation and Reinforcement Learningcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 24; 2024: 31; 2025: 36; 2026: 18.

113 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  3. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  4. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 135
  5. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  6. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  7. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  8. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 71
  9. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  10. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  11. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 59
  12. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
38 more
  1. 2022End-to-End Affordance Learning for Robotic Manipulationcited by 54
  2. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  3. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 50
  4. 2024Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulationcited by 45
  5. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  6. 2024RIME: Robust Preference-based Reinforcement Learning with Noisy Preferencescited by 37
  7. 2024PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Taskscited by 37
  8. 2023Stabilize to Act: Learning to Coordinate for Bimanual Manipulationcited by 36
  9. 2024Twisting Lids Off with Two Handscited by 36
  10. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  11. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 33
  12. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  13. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 29
  14. 2023OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Controlcited by 28
  15. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 27
  16. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 24
  17. 2023GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Graspingcited by 23
  18. 2024Lessons from Learning to Spin "Pens"cited by 22
  19. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  20. 2024InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulationcited by 21
  21. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21
  22. 2024Bi-VLA: Vision-Language-Action Model-Based System for Bimanual Robotic Dexterous Manipulationscited by 20
  23. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  24. 2022DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulationcited by 18
  25. 2023Grasp Multiple Objects with One Handcited by 18
  26. 2024Discrete Policy: Learning Disentangled Action Space for Multi-Task Robotic Manipulationcited by 17
  27. 2024Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Taskscited by 16
  28. 2024ScrewMimic: Bimanual Imitation from Human Videos with Screw Space Projectioncited by 16
  29. 2025CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-Worldcited by 16
  30. 2024Learning Visual Quadrupedal Loco-Manipulation from Demonstrationscited by 15
  31. 2024DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulationcited by 15
  32. 2025Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Modelscited by 15
  33. 2024Point Cloud Models Improve Visual Robustness in Robotic Learnerscited by 13
  34. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  35. 2023Learning a Universal Human Prior for Dexterous Manipulation from Human Preferencecited by 10
  36. 2024Large Language Models for Orchestrating Bimanual Robotscited by 10
  37. 2023Towards Generalist Robots: A Promising Paradigm via Generative Simulationcited by 9
  38. 2024A Review of Nine Physics Engines for Reinforcement Learning Researchcited by 9

50 most cited of 113 All 113 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Human-level Dexterity via Robot Learning
  2. 2026Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulation ↗
  3. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning
  4. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
  5. 2025Dexterous Manipulation through Imitation Learning: A Survey
  6. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  7. 2025Learning Bimanual Manipulation via Action Chunking and Inter-Arm Coordination with Transformers
  8. 2025DexterityGen: Foundation Controller for Unprecedented Dexterity