# RHOBIN Challenge: Reconstruction of Human Object Interaction arXiv: https://arxiv.org/abs/2401.04143 ยท HTML: https://mlsanity.com/paper/2401.04143/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 65 of its 75 references that are in the map. - First posted on arXiv 7 Jan 2024. ## Builds on (most cited first, 30 of 65) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019) - [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019) - [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019) - [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019) - [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018) - [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017) - [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019) - [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017) - [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016) - [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019) - [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019) - [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020) - [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019) - [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020) - [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018) - [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019) - [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020) - [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019) - [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020) - [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019) - [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018) - [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021) - [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018) - [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022) - [Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation](https://mlsanity.com/paper/1908.07433/) (2019) - [Monocular, One-stage, Regression of Multiple 3D People](https://mlsanity.com/paper/2008.12272/) (2020) - [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022) - [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021) - [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020) ## Built on by (most cited first, 10 of 10) - [Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer](https://mlsanity.com/paper/2404.04819/) (2024) - [InterTrack: Tracking Human Object Interaction without Object Templates](https://mlsanity.com/paper/2408.13953/) (2024) - [Survey on Modeling of Human-made Articulated Objects](https://mlsanity.com/paper/2403.14937/) (2024) - [Learning Dense Hand Contact Estimation from Imbalanced Data](https://mlsanity.com/paper/2505.11152/) (2025) - [CARI4D: Category Agnostic 4D Reconstruction of Human-Object Interaction](https://mlsanity.com/paper/2512.11988/) (2025) - [LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Model](https://mlsanity.com/paper/2603.27449/) (2026) - [End-to-End HOI Reconstruction Transformer with Graph-based Encoding](https://mlsanity.com/paper/2503.06012/) (2025) - [Shoe Style-Invariant and Ground-Aware Learning for Dense Foot Contact Estimation](https://mlsanity.com/paper/2511.22184/) (2025) - [TeHOR: Text-Guided 3D Human and Object Reconstruction with Textures](https://mlsanity.com/paper/2602.19679/) (2026) - [Hoi3DGen: Generating High-Quality Human-Object-Interactions in 3D](https://mlsanity.com/paper/2603.12126/) (2026) ## Abstract (arXiv) Modeling the interaction between humans and objects has been an emerging research direction in recent years. Capturing human-object interaction is however a very challenging task due to heavy occlusion and complex dynamics, which requires understanding not only 3D human pose, and object pose but also the interaction between them. Reconstruction of 3D humans and objects has been two separate research fields in computer vision for a long time. We hence proposed the first RHOBIN challenge: reconstruction of human-object interactions in conjunction with the RHOBIN workshop. It was aimed at bringing the research communities of human and object reconstruction as well as interaction modeling together to discuss techniques and exchange ideas. Our challenge consists of three tracks of 3D reconstruction from monocular RGB images with a focus on dealing with challenging interaction scenarios. Our challenge attracted more than 100 participants with more than 300 submissions, indicating the broad interest in the research communities. This paper describes the settings of our challenge and discusses the winning methods of each track in more detail. We observe that the human reconstruction task is becoming mature even under heavy occlusion settings while object pose estimation and joint reconstruction remain challenging tasks. With the growing interest in interaction modeling, we hope this report can provide useful insights and foster future research in this direction. Our workshop website can be found at hrefhttps://rhobin-challenge.github.io/https://rhobin-challenge.github.io/.