mlsanityData to 14 Oct 2026

ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning

arXiv 2503.21860 ↗First posted 27 Mar 2025cs.RO cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

Human hands play a central role in interacting, motivating increasing research in dexterous robotic manipulation. Data-driven embodied AI algorithms demand precise, large-scale, human-like manipulation sequences, which are challenging to obtain with conventional reinforcement learning or real-world teleoperation. To address this, we introduce ManipTrans, a novel two-stage method for efficiently transferring human bimanual skills to dexterous robotic hands in simulation. ManipTrans first pre-trains a generalist trajectory imitator to mimic hand motion, then fine-tunes a specific residual module under interaction constraints, enabling efficient learning and accurate execution of complex bimanual tasks. Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging ManipTrans, we transfer multiple hand-object datasets to robotic hands, creating DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K episodes of robotic manipulation and is easily extensible, facilitating further policy training for dexterous hands and enabling real-world deployments.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 118 of its 144 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

118 of 144 references.

References · 118

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  9. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  10. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  11. 2021Implicit Behavioral Cloningcited by 452
  12. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
38 more
  1. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  2. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  3. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  4. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  5. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  6. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 312
  7. 2023Reconstructing Hands in 3D with Transformerscited by 306
  8. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  9. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  10. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  11. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  12. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
  13. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  14. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  15. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
  16. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  17. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 204
  18. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189
  19. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
  20. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  21. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 183
  22. 2021A System for General In-Hand Object Re-Orientationcited by 165
  23. 2018Residual Policy Learningcited by 164
  24. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  25. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  26. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  27. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  28. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 147
  29. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  30. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  31. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 141
  32. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  33. 2022DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulationcited by 137
  34. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 131
  35. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 130
  36. 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 128
  37. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 124
  38. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123

50 most cited of 118 All 118 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 52.

66 papers.

Sort:

By area:

  1. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  2. 2025ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Modelscited by 43
  3. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 41
  4. 2025DexMachina: Functional Retargeting for Bimanual Dexterous Manipulationcited by 37
  5. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 32
  6. 2026CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videoscited by 31
  7. 2025DreamControl: Human-Inspired Whole-Body Humanoid Control for Scene Interaction via Guided Diffusioncited by 15
  8. 2026SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulationcited by 15
  9. 2026Do as I Do: Dexterous Manipulation Data from Everyday Human Videoscited by 13
  10. 2025Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Explorationcited by 11
  11. 2026Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstructioncited by 11
  12. 2025Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understandingcited by 10
38 more
  1. 2025DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videoscited by 10
  2. 2025Scaffolding Dexterous Manipulation with Vision-Language Modelscited by 9
  3. 2026Dexora: Open-source VLA for High-DoF Bimanual Dexteritycited by 9
  4. 2026EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videoscited by 7
  5. 2026DexHiL: A Human-in-the-Loop Framework for Vision-Language-Action Model Post-Training in Dexterous Manipulationcited by 6
  6. 2026TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulationcited by 6
  7. 2026Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstrationcited by 6
  8. 2025Uni-Hand: Universal Hand Motion Forecasting in Egocentric Viewscited by 5
  9. 2026ObjectForesight: Predicting Future 3D Object Trajectories from Human Videoscited by 5
  10. 2026LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisitioncited by 5
  11. 2026Human Universal Graspingcited by 5
  12. 2025PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attentioncited by 4
  13. 2026A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulationcited by 4
  14. 2026UniHM: Unified Dexterous Hand Manipulation with Vision Language Modelcited by 3
  15. 2026FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulationcited by 3
  16. 2026PAM: A Pose-Appearance-Motion Engine for Sim-to-Real HOI Video Generationcited by 3
  17. 2026AnyHand: A Large-Scale Synthetic Dataset for RGB(-D) Hand Pose Estimationcited by 2
  18. 2026StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Videocited by 2
  19. 2026Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulationcited by 2
  20. 2026Dex2HOI: Dexterous Bimanual Two-Object Interaction Generationcited by 2
  21. 2026EgoAERO: Learning Dexterous Manipulation from a Single Egocentric Video without Object Assetscited by 2
  22. 2026CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulationcited by 2
  23. 2026UniCross: Unified Cross-Skill Dexterous Manipulation Synthesiscited by 2
  24. 2025UniBYD: A Unified Framework for Learning Robotic Manipulation Across Embodiments Beyond Imitation of Human Demonstrationscited by 1
  25. 2026Grasp-and-Lift: Executable 3D Hand-Object Interaction Reconstruction via Physics-in-the-Loop Optimizationcited by 1
  26. 2026THOM: Generating Physically Plausible Hand-Object Meshes From Textcited by 1
  27. 2026Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?cited by 1
  28. 2026MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Predictioncited by 1
  29. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 1
  30. 2026DexMani: Human-Derived Manipulability Guidance for Dexterous Rotationcited by 1
  31. 2026DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Librarycited by 1
  32. 2025D3Grasp: Diverse and Deformable Dexterous Grasping for General Objectscited by 0
  33. 2026FSAG: Enhancing Human-to-Dexterous-Hand Finger-Specific Affordance Grounding via Diffusion Modelscited by 0
  34. 2026DuetHOI: Language-Guided Bimanual Hand--Object Motion Generation with Articulation Planning and Contact Refinementcited by 0
  35. 2026TransDex: Pre-training Visuo-Tactile Policy with Point Cloud Reconstruction for Dexterous Manipulation of Transparent Objectscited by 0
  36. 2026Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catchingcited by 0
  37. 2026Recovering Physically Plausible Human-Object Interactions from Monocular Videoscited by 0
  38. 2026UniDexTok: A Unified Dexterous Hand Tokenizer from Real Datacited by 0

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Dexterous Manipulation through Imitation Learning: A Survey
  2. 2026DemoBot: Efficient Learning of Bimanual Manipulation with Dexterous Hands From Third-Person Human Videos
  3. 2025DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
  4. 2026Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations
  5. 2026DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations
  6. 2026DexSynRefine: Synthesizing and Refining Human-Object Interaction Motion for Physically Feasible Dexterous Robot Actions
  7. 2023Stabilize to Act: Learning to Coordinate for Bimanual Manipulation
  8. 2026DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations ↗