mlsanityData to 7 Oct 2026

HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments

arXiv 2604.14944 ↗First posted 16 Apr 2026cs.RO cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

We present HRDexDB, a real-world 4D dexterous grasping dataset capturing 3D hand-object interaction trajectories over time across five embodiments. The dataset comprises 3.2K trials over 100 diverse objects. Using a synchronized multi-camera system and an integrated reconstruction pipeline, HRDexDB provides multi-view and egocentric RGB observations, 3D hand geometry, robot states, and object 6D pose trajectories, together with success/failure annotations. Human and robotic hands interact with shared objects, enabling the study of embodiment-dependent grasp strategies and contact patterns. We demonstrate the dataset's utility through human-to-robot contact map transfer, visual robot-object contact estimation, and retrieval-assisted grasping. Together, these results establish HRDexDB as a resource for studying and learning dexterous interactions across human and robotic embodiments.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

45 of 53 references.

References · 45

  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  4. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  5. 2025SAM 3: Segment Anything with Conceptscited by 835
  6. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  7. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  8. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  9. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  10. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  11. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  12. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
33 more
  1. 2023Reconstructing Hands in 3D with Transformerscited by 298
  2. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  3. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  4. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  5. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  6. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  7. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  8. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  9. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 145
  10. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 136
  11. 2022DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulationcited by 129
  12. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  13. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  14. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 85
  15. 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 78
  16. 2025RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulationcited by 72
  17. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 59
  18. 2022GenDexGrasp: Generalizable Dexterous Graspingcited by 54
  19. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 34
  20. 2024RealDex: Towards Human-like Grasping for Robotic Dexterous Handcited by 33
  21. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 32
  22. 2024HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interactioncited by 25
  23. 2025Human2Robot: Learning Robot Actions from Paired Human-Robot Videoscited by 24
  24. 2022PressureVision: Estimating Hand Pressure from a Single RGB Imagecited by 23
  25. 2024BODex: Scalable and Efficient Robotic Dexterous Grasp Synthesis Using Bilevel Optimizationcited by 18
  26. 2025RobustDexGrasp: Robust Dexterous Grasping of General Objectscited by 16
  27. 2025Learning Dense Hand Contact Estimation from Imbalanced Datacited by 12
  28. 2024DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexteritycited by 11
  29. 2025GoTrack: Generic 6DoF Object Pose Refinement and Trackingcited by 10
  30. 2025CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representationscited by 9
  31. 2026cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF Robotscited by 8
  32. 2025Enhancing Tactile-based Reinforcement Learning for Robotic Controlcited by 5
  33. 2025Visuo-Tactile Object Pose Estimation for a Multi-Finger Robot Hand with Low-Resolution In-Hand Tactile Sensingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026PhiZero: A World Model Built Around Physical Languagecited by 2
  2. 2026Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Handscited by 2