# Physics-aware Hand-object Interaction Denoising

arXiv: https://arxiv.org/abs/2405.11481 · HTML: https://mlsanity.com/paper/2405.11481/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 19 May 2024.

## Builds on (most cited first, 20 of 20)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging](https://mlsanity.com/paper/1904.06830/) (2019)
- [Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction](https://mlsanity.com/paper/2004.13449/) (2020)
- [ContactOpt: Optimizing Contact to Improve Grasps](https://mlsanity.com/paper/2104.07267/) (2021)
- [D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions](https://mlsanity.com/paper/2112.03028/) (2021)
- [Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera](https://mlsanity.com/paper/2106.08059/) (2021)
- [TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement](https://mlsanity.com/paper/2205.07982/) (2022)
- [GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping](https://mlsanity.com/paper/1811.10983/) (2018)
- [Towards unconstrained joint hand-object reconstruction from RGB videos](https://mlsanity.com/paper/2108.07044/) (2021)
- [Learning to Use Chopsticks in Diverse Gripping Styles](https://mlsanity.com/paper/2205.14313/) (2022)
- [HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/2004.01588/) (2020)
- [Physical Interaction: Reconstructing Hand-object Interactions with Physics](https://mlsanity.com/paper/2209.10833/) (2022)
- [Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild](https://mlsanity.com/paper/2209.12009/) (2022)
- [Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model](https://mlsanity.com/paper/1602.03860/) (2016)
- [MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation](https://mlsanity.com/paper/2012.03206/) (2020)

## Built on by (most cited first, 8 of 8)

- [TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions](https://mlsanity.com/paper/2412.06334/) (2024)
- [OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Model](https://mlsanity.com/paper/2505.18947/) (2025)
- [Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model](https://mlsanity.com/paper/2412.20657/) (2024)
- [Diffusion-based 3D Hand Motion Recovery with Intuitive Physics](https://mlsanity.com/paper/2508.01835/) (2025)
- [PAD-Hand: Physics-Aware Diffusion for Hand Motion Recovery](https://mlsanity.com/paper/2603.26068/) (2026)
- [Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer](https://mlsanity.com/paper/2607.28394/) (2026)
- [DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstruction](https://mlsanity.com/paper/2609.36454/) (2026)
- [StableGrasp: Reconstructing Physically Stable Human Hand Grasps from Single Images](https://mlsanity.com/paper/2610.09195/) (2026)

## Abstract (arXiv)

The credibility and practicality of a reconstructed hand-object interaction sequence depend largely on its physical plausibility. However, due to high occlusions during hand-object interaction, physical plausibility remains a challenging criterion for purely vision-based tracking methods. To address this issue and enhance the results of existing hand trackers, this paper proposes a novel physically-aware hand motion de-noising method. Specifically, we introduce two learned loss terms that explicitly capture two crucial aspects of physical plausibility: grasp credibility and manipulation feasibility. These terms are used to train a physically-aware de-noising network. Qualitative and quantitative experiments demonstrate that our approach significantly improves both fine-grained physical plausibility and overall pose accuracy, surpassing current state-of-the-art de-noising methods.