Kinematics-based 3D Human-Object Interaction Reconstruction from Single View
Not on the map. Why?
Abstract (arXiv)
Reconstructing 3D human-object interaction (HOI) from single-view RGB images is challenging due to the absence of depth information and potential occlusions. Existing methods simply predict the body poses merely rely on network training on some indoor datasets, which cannot guarantee the rationality of the results if some body parts are invisible due to occlusions that appear easily. Inspired by the end-effector localization task in robotics, we propose a kinematics-based method that can drive the joints of human body to the human-object contact regions accurately. After an improved forward kinematics algorithm is proposed, the Multi-Layer Perceptron is introduced into the solution of inverse kinematics process to determine the poses of joints, which achieves precise results than the commonly-used numerical methods in robotics. Besides, a Contact Region Recognition Network (CRRNet) is also proposed to robustly determine the contact regions using a single-view video. Experimental results demonstrate that our method outperforms the state-of-the-art on benchmark BEHAVE. Additionally, our approach shows good portability and can be seamlessly integrated into other methods for optimizations.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 19 Jul 2024.
Builds on
26 of 39 references.
References · 26
- 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
- 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
- 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
- 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
- 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
- 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
- 2023Object Motion Guided Human Motion Synthesiscited by 206
- 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
- 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
- 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
- 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
- 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 127
14 more
- 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
- 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 90
- 2022Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformerscited by 81
- 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 72
- 2019Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsensecited by 71
- 2023Visibility Aware Human-Object Interaction Tracking from Single RGB Cameracited by 58
- 2020Holistic 3D Human and Scene Mesh Estimation from Single View Imagescited by 54
- 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
- 2023ContactGen: Generative Contact Modeling for Grasp Generationcited by 52
- 2023HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recoverycited by 48
- 2023Synthesizing Physically Plausible Human Motions in 3D Scenescited by 47
- 2022Interacting Hand-Object Pose Estimation via Dense Mutual Attentioncited by 22
- 2023Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Imagescited by 10
- 2022Stability-driven Contact Reconstruction From Monocular Color Imagescited by 8
Built on by
Citing papers per year: 2024: 1.
1 paper.