mlsanityData to 7 Oct 2026

Physical Interaction: Reconstructing Hand-object Interactions with Physics

arXiv 2209.10833 ↗First posted 22 Sep 2022cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Single view-based reconstruction of hand-object interaction is challenging due to the severe observation missing caused by occlusions. This paper proposes a physics-based method to better solve the ambiguities in the reconstruction. It first proposes a force-based dynamic model of the in-hand object, which not only recovers the unobserved contacts but also solves for plausible contact forces. Next, a confidence-based slide prevention scheme is proposed, which combines both the kinematic confidences and the contact forces to jointly model static and sliding contact motion. Qualitative and quantitative experiments show that the proposed technique reconstructs both physically plausible and more accurate hand-object interaction and estimates plausible contact forces in real-time with a single RGBD sensor.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 21 of its 43 references that are in the map. First posted on arXiv 22 Sep 2022.

Builds on

21 of 43 references.

References · 21

  1. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  2. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  3. 2022Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensorscited by 144
  4. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 139
  5. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
  6. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  7. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 111
  8. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  9. 2020Contact and Human Dynamics from Monocular Videocited by 94
  10. 2019Ego-Pose Estimation and Forecasting as Real-Time PD Controlcited by 87
  11. 2021Neural Monocular 3D Human Motion Capture with Physical Awarenesscited by 70
  12. 2016Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Inputcited by 56
9 more
  1. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  2. 2019Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Videocited by 48
  3. 2020Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effectscited by 40
  4. 2020Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimationcited by 34
  5. 2022Learning to Use Chopsticks in Diverse Gripping Stylescited by 29
  6. 2018SFV: Reinforcement Learning of Physical Skills from Videoscited by 20
  7. 2020Weakly-supervised Learning of Human Dynamicscited by 13
  8. 2017Back to RGB: 3D tracking of hands and hand-object interactions based on short-baseline stereocited by 8
  9. 2021Physically Plausible Pose Refinement using Fully Differentiable Forcescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 4; 2025: 3; 2026: 4.

19 papers.

Sort:

  1. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  2. 2023Decaf: Monocular Deformation Capture for Face and Hand Interactionscited by 17
  3. 2024Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamicscited by 15
  4. 2023MACS: Mass Conditioned 3D Hand and Object Motion Synthesiscited by 13
  5. 2023Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Imagescited by 10
  6. 2023Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstructioncited by 10
  7. 2023Few-Shot Physically-Aware Articulated Mesh Generation via Hierarchical Deformationcited by 8
  8. 2023DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulationcited by 8
  9. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  10. 2024Physics-aware Hand-object Interaction Denoisingcited by 7
  11. 2024Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with Physicscited by 6
  12. 2026PAD-Hand: Physics-Aware Diffusion for Hand Motion Recoverycited by 4
7 more
  1. 2023HandyPriors: Physically Consistent Perception of Hand-Object Interactions with Differentiable Priorscited by 3
  2. 2025FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidancecited by 2
  3. 2026Grasp in Gaussians: Fast Monocular Reconstruction of Dynamic Hand-Object Interactionscited by 2
  4. 2025Human Motion Prediction, Reconstruction, and Generationcited by 1
  5. 2025VPHO: Joint Visual-Physical Cue Learning and Aggregation for Hand-Object Pose Estimationcited by 0
  6. 2026DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulationcited by 0
  7. 2026DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstructioncited by 0