# Holistic 3D Human and Scene Mesh Estimation from Single View Images

arXiv: https://arxiv.org/abs/2012.01591 · HTML: https://mlsanity.com/paper/2012.01591/

## Key facts

- Cited by 54 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 2 Dec 2020.

## Built on by (most cited first, 30 of 54)

- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Capturing and Inferring Dense Full-Body Human-Scene Contact](https://mlsanity.com/paper/2206.09553/) (2022)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices](https://mlsanity.com/paper/2112.07642/) (2021)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [3D Human Pose Estimation via Intuitive Physics](https://mlsanity.com/paper/2303.18246/) (2023)
- [CHORE: Contact, Human and Object REconstruction from a single RGB image](https://mlsanity.com/paper/2204.02445/) (2022)
- [Full-Body Articulated Human-Object Interaction](https://mlsanity.com/paper/2212.10621/) (2022)
- [Visibility Aware Human-Object Interaction Tracking from Single RGB Camera](https://mlsanity.com/paper/2303.16479/) (2023)
- [Human-Aware Object Placement for Visual Environment Reconstruction](https://mlsanity.com/paper/2203.03609/) (2022)
- [DECO: Dense Estimation of 3D Human-Scene Contact In The Wild](https://mlsanity.com/paper/2309.15273/) (2023)
- [HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment](https://mlsanity.com/paper/2404.00299/) (2024)
- [Detecting Human-Object Contact in Images](https://mlsanity.com/paper/2303.03373/) (2023)
- [D3D-HOI: Dynamic 3D Human-Object Interactions from Videos](https://mlsanity.com/paper/2108.08420/) (2021)
- [Object pop-up: Can we infer 3D objects and their poses from human interactions alone?](https://mlsanity.com/paper/2306.00777/) (2023)
- [Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views](https://mlsanity.com/paper/2304.06024/) (2023)
- [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023)
- [The One Where They Reconstructed 3D Humans and Environments in TV Shows](https://mlsanity.com/paper/2207.14279/) (2022)
- [Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors](https://mlsanity.com/paper/2209.02485/) (2022)
- [Scene-aware Egocentric 3D Human Pose Estimation](https://mlsanity.com/paper/2212.11684/) (2022)
- [Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion](https://mlsanity.com/paper/2205.02830/) (2022)
- [F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions](https://mlsanity.com/paper/2407.12435/) (2024)
- [ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors](https://mlsanity.com/paper/2308.12969/) (2023)
- [InterTrack: Tracking Human Object Interaction without Object Templates](https://mlsanity.com/paper/2408.13953/) (2024)
- [PICO: Reconstructing 3D People In Contact with Objects](https://mlsanity.com/paper/2504.17695/) (2025)
- [Scene-Aware 3D Multi-Human Motion Capture from a Single Camera](https://mlsanity.com/paper/2301.05175/) (2023)
- [MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes](https://mlsanity.com/paper/2208.08439/) (2022)
- [ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image](https://mlsanity.com/paper/2305.16411/) (2023)
- [TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactions](https://mlsanity.com/paper/2412.06334/) (2024)
- [A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signals](https://mlsanity.com/paper/2404.04890/) (2024)

## Abstract (arXiv)

The 3D world limits the human body pose and the human body pose conveys information about the surrounding objects. Indeed, from a single image of a person placed in an indoor scene, we as humans are adept at resolving ambiguities of the human pose and room layout through our knowledge of the physical laws and prior perception of the plausible object and human poses. However, few computer vision models fully leverage this fact. In this work, we propose an end-to-end trainable model that perceives the 3D scene from a single RGB image, estimates the camera pose and the room layout, and reconstructs both human body and object meshes. By imposing a set of comprehensive and sophisticated losses on all aspects of the estimations, we show that our model outperforms existing human body mesh methods and indoor scene reconstruction methods. To the best of our knowledge, this is the first model that outputs both object and human predictions at the mesh level, and performs joint optimization on the scene and human poses.