# Hand-Object Contact Consistency Reasoning for Human Grasps Generation

arXiv: https://arxiv.org/abs/2104.03304 · HTML: https://mlsanity.com/paper/2104.03304/

## Key facts

- Cited by 160 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 7 Apr 2021.

## Builds on (most cited first, 29 of 29)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [Grasping Field: Learning Implicit Representations for Human Grasps](https://mlsanity.com/paper/2008.04451/) (2020)
- [Semantic Photo Manipulation with a Generative Image Prior](https://mlsanity.com/paper/2005.07727/) (2020)
- [ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging](https://mlsanity.com/paper/1904.06830/) (2019)
- [H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions](https://mlsanity.com/paper/1904.05349/) (2019)
- [Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes](https://mlsanity.com/paper/2012.05522/) (2020)
- [Domain Adaptation for Visual Applications: A Comprehensive Survey](https://mlsanity.com/paper/1702.05374/) (2017)
- [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017)
- [Capturing Hands in Action using Discriminative Salient Points and Physics Simulation](https://mlsanity.com/paper/1506.02178/) (2015)
- [HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation](https://mlsanity.com/paper/2004.00060/) (2020)
- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact](https://mlsanity.com/paper/1904.03754/) (2019)
- [Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments](https://mlsanity.com/paper/1903.05690/) (2019)
- [Online Model Distillation for Efficient Video Inference](https://mlsanity.com/paper/1812.02699/) (2018)
- [Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effects](https://mlsanity.com/paper/2003.12045/) (2020)
- [SuctionNet-1Billion: A Large-Scale Benchmark for Suction Grasping](https://mlsanity.com/paper/2103.12311/) (2021)
- [Generalized Feedback Loop for Joint Hand-Object Pose Estimation](https://mlsanity.com/paper/1903.10883/) (2019)

## Built on by (most cited first, 30 of 160)

- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion](https://mlsanity.com/paper/2308.16905/) (2023)
- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation](https://mlsanity.com/paper/2210.02697/) (2022)
- [OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction](https://mlsanity.com/paper/2203.15709/) (2022)
- [EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos](https://mlsanity.com/paper/2507.12440/) (2025)
- [IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions](https://mlsanity.com/paper/2212.07555/) (2022)
- [UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy](https://mlsanity.com/paper/2303.00938/) (2023)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions](https://mlsanity.com/paper/2112.03028/) (2021)
- [Affordance Diffusion: Synthesizing Hand-Object Interactions](https://mlsanity.com/paper/2303.12538/) (2023)
- [Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos](https://mlsanity.com/paper/2507.15597/) (2025)
- [SAGA: Stochastic Whole-Body Grasping with Contact](https://mlsanity.com/paper/2112.10103/) (2021)
- [Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction](https://mlsanity.com/paper/2404.00562/) (2024)
- [OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion](https://mlsanity.com/paper/2403.19417/) (2024)
- [Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance](https://mlsanity.com/paper/2403.18036/) (2024)
- [Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications](https://mlsanity.com/paper/2208.03826/) (2022)
- [ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis](https://mlsanity.com/paper/2109.05488/) (2021)
- [MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image](https://mlsanity.com/paper/2112.02753/) (2021)
- [Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clips](https://mlsanity.com/paper/2309.05663/) (2023)
- [TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement](https://mlsanity.com/paper/2205.07982/) (2022)
- [ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning](https://mlsanity.com/paper/2503.21860/) (2025)
- [Human-Object Interaction from Human-Level Instructions](https://mlsanity.com/paper/2406.17840/) (2024)
- [GraspXL: Generating Grasping Motions for Diverse Objects at Scale](https://mlsanity.com/paper/2403.19649/) (2024)
- [Human-Aware Object Placement for Visual Environment Reconstruction](https://mlsanity.com/paper/2203.03609/) (2022)
- [GenDexGrasp: Generalizable Dexterous Grasping](https://mlsanity.com/paper/2210.00722/) (2022)
- [CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesis](https://mlsanity.com/paper/2303.15469/) (2023)
- [ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation](https://mlsanity.com/paper/2309.03891/) (2023)
- [ContactGen: Generative Contact Modeling for Grasp Generation](https://mlsanity.com/paper/2310.03740/) (2023)

## Abstract (arXiv)

While predicting robot grasps with parallel jaw grippers have been well studied and widely applied in robot manipulation tasks, the study on natural human grasp generation with a multi-finger hand remains a very challenging problem. In this paper, we propose to generate human grasps given a 3D object in the world. Our key observation is that it is crucial to model the consistency between the hand contact points and object contact regions. That is, we encourage the prior hand contact points to be close to the object surface and the object common contact regions to be touched by the hand at the same time. Based on the hand-object contact consistency, we design novel objectives in training the human grasp generation model and also a new self-supervised task which allows the grasp generation network to be adjusted even during test time. Our experiments show significant improvement in human grasp generation over state-of-the-art approaches by a large margin. More interestingly, by optimizing the model during test time with the self-supervised task, it helps achieve larger gain on unseen and out-of-domain objects. Project page: https://hwjiang1510.github.io/GraspTTA/