# RealDex: Towards Human-like Grasping for Robotic Dexterous Hand

arXiv: https://arxiv.org/abs/2402.13853 · HTML: https://mlsanity.com/paper/2402.13853/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [Synthesis of Compositional Animations from Textual Descriptions](https://mlsanity.com/paper/2103.14675/) (2021)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping](https://mlsanity.com/paper/2112.11454/) (2021)
- [DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation](https://mlsanity.com/paper/2210.02697/) (2022)
- [OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction](https://mlsanity.com/paper/2203.15709/) (2022)
- [Capturing Hands in Action using Discriminative Salient Points and Physics Simulation](https://mlsanity.com/paper/1506.02178/) (2015)
- [D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions](https://mlsanity.com/paper/2112.03028/) (2021)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [SAGA: Stochastic Whole-Body Grasping with Contact](https://mlsanity.com/paper/2112.10103/) (2021)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF](https://mlsanity.com/paper/2210.06575/) (2022)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception](https://mlsanity.com/paper/2109.05441/) (2021)
- [TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline](https://mlsanity.com/paper/2202.08471/) (2022)
- [Learning Generalizable Dexterous Manipulation from Human Grasp Affordance](https://mlsanity.com/paper/2204.02320/) (2022)
- [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021)
- [Deep Differentiable Grasp Planner for High-DOF Grippers](https://mlsanity.com/paper/2002.01530/) (2020)
- [DDGC: Generative Deep Dexterous Grasping in Clutter](https://mlsanity.com/paper/2103.04783/) (2021)
- [Learning High-DOF Reaching-and-Grasping via Dynamic Representation of Gripper-Object Interaction](https://mlsanity.com/paper/2204.13998/) (2022)
- [Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation](https://mlsanity.com/paper/2210.13638/) (2022)
- [Human-centric Scene Understanding for 3D Large-scale Scenarios](https://mlsanity.com/paper/2307.14392/) (2023)
- [DVGG: Deep Variational Grasp Generation for Dextrous Manipulation](https://mlsanity.com/paper/2211.11154/) (2022)

## Built on by (most cited first, 30 of 34)

- [SemGrasp: Semantic Grasp Generation via Language Aligned Discretization](https://mlsanity.com/paper/2404.03590/) (2024)
- [Pack and Force Your Memory: Long-form and Consistent Video Generation](https://mlsanity.com/paper/2510.01784/) (2025)
- [DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awareness](https://mlsanity.com/paper/2503.08257/) (2025)
- [Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation](https://mlsanity.com/paper/2506.17198/) (2025)
- [Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy](https://mlsanity.com/paper/2504.18829/) (2025)
- [T-Rex: Tactile-Reactive Dexterous Manipulation](https://mlsanity.com/paper/2606.17055/) (2026)
- [Multi-GraspLLM: A Multimodal LLM for Multi-Hand Semantic Guided Grasp Generation](https://mlsanity.com/paper/2412.08468/) (2024)
- [Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context](https://mlsanity.com/paper/2510.20229/) (2025)
- [FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens](https://mlsanity.com/paper/2506.01583/) (2025)
- [CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representations](https://mlsanity.com/paper/2509.24661/) (2025)
- [Dexora: Open-source VLA for High-DoF Bimanual Dexterity](https://mlsanity.com/paper/2605.18722/) (2026)
- [FastGrasp: Efficient Grasp Synthesis with Diffusion](https://mlsanity.com/paper/2411.14786/) (2024)
- [EvolvingGrasp: Evolutionary Grasp Generation via Efficient Preference Alignment](https://mlsanity.com/paper/2503.14329/) (2025)
- [RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands](https://mlsanity.com/paper/2408.11048/) (2024)
- [HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments](https://mlsanity.com/paper/2604.14944/) (2026)
- [Bimanual Grasp Synthesis for Dexterous Robot Hands](https://mlsanity.com/paper/2411.15903/) (2024)
- [Grasping a Handful: Sequential Multi-Object Dexterous Grasp Generation](https://mlsanity.com/paper/2503.22370/) (2025)
- [Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation](https://mlsanity.com/paper/2511.01276/) (2025)
- [Dexterous Grasping with Real-World Robotic Reinforcement Learning](https://mlsanity.com/paper/2503.04014/) (2025)
- [UniMorphGrasp: Diffusion Model with Morphology-Awareness for Cross-Embodiment Dexterous Grasp Generation](https://mlsanity.com/paper/2602.00915/) (2026)
- [AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis](https://mlsanity.com/paper/2603.08021/) (2026)
- [SFHand: Learning Embodied Manipulation by Streaming Egocentric 3D Hand Forecasting](https://mlsanity.com/paper/2511.18127/) (2025)
- [VacuumVLA: Boosting VLA Capabilities via a Unified Suction and Gripping Tool for Complex Robotic Manipulation](https://mlsanity.com/paper/2511.21557/) (2025)
- [Generate, Transfer, Adapt: Learning Functional Dexterous Grasping from a Single Human Demonstration](https://mlsanity.com/paper/2601.05243/) (2026)
- [EFF-Grasp: Energy-Field Flow Matching for Physics-Aware Dexterous Grasp Generation](https://mlsanity.com/paper/2603.16151/) (2026)
- [AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection](https://mlsanity.com/paper/2606.23689/) (2026)
- [GenHeld: Generating and Editing Handheld Objects](https://mlsanity.com/paper/2406.05059/) (2024)
- [TransDex: Pre-training Visuo-Tactile Policy with Point Cloud Reconstruction for Dexterous Manipulation of Transparent Objects](https://mlsanity.com/paper/2603.13869/) (2026)
- [FastGrasp: Learning-based Whole-Body Control Method for Fast Dexterous Grasping with Mobile Manipulators](https://mlsanity.com/paper/2604.12879/) (2026)
- [SECOND-Grasp: Semantic Contact-guided Dexterous Grasping](https://mlsanity.com/paper/2605.13117/) (2026)

## Abstract (arXiv)

In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data. Utilizing a teleoperation system, we seamlessly synchronize human-robot hand poses in real time. This collection of human-like motions is crucial for training dexterous hands to mimic human movements more naturally and precisely. RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios. Moreover, we introduce a cutting-edge dexterous grasping motion generation framework, which aligns with human experience and enhances real-world applicability through effectively utilizing Multimodal Large Language Models. Extensive experiments have demonstrated the superior performance of our method on RealDex and other open datasets. The complete dataset and code will be made available upon the publication of this work.