Scene and Human in One World: Reconstruction in a Feedforward Pass
Not on the map. Why?
Abstract (arXiv)
Reconstructing humans in dynamic scenes from moving monocular cameras remains challenging due to scale ambiguity, human-scene misalignment, and occlusion interference. Rather than treating human mesh recovery and scene reconstruction as separate tasks, we believe that accurate human-scene reconstruction requires the two tasks to mutually inform each other: parametric human models offer semantic structure and metric-scale priors, while scene geometry provides spatial context for human localization and alignment. Built on this insight, we introduce SHOW, a mask-promptable human mesh recovery framework that couples feed-forward 3D scene reconstruction with Human Mesh Recovery in a unified metric space. SHOW injects human semantics and scale priors from parametric human models into normalized point-map prediction, enabling metric-scale scene reconstruction from inherently scale-ambiguous monocular input. In turn, the recovered scene geometry constrains human mesh estimation, encouraging spatially consistent human placement and improved human-scene alignment. To handle complex multi-person and cluttered scenes, SHOW further incorporates a promptable masking mechanism that enables flexible target-human selection while suppressing background distractions and occlusion interference. Through joint training, the model learns both human-aware geometric features and geometry-constrained human features, producing aligned metric-scale reconstructions from monocular human-centric videos. Extensive experiments demonstrate that SHOW improves metric-scale consistency, human-scene alignment, and reconstruction accuracy under challenging camera motion, occlusion, and cluttered backgrounds.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 26 Jun 2026.
Builds on
22 of 33 references.
References · 22
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
- 2017End-to-end Recovery of Human Shape and Posecited by 885
- 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
- 2025Continuous 3D Perception Model with Persistent Statecited by 548
- 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
- 2018Learning 3D Human Dynamics from Videocited by 316
- 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
- 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
- 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 106
- 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 95
- 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
- 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
10 more
- 2026SAM 3D Body: Robust Full-Body Human Mesh Recoverycited by 69
- 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 59
- 2023Visibility Aware Human-Object Interaction Tracking from Single RGB Cameracited by 58
- 2025Human3R: Everyone Everywhere All at Oncecited by 37
- 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
- 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 23
- 2021Pose2Room: Understanding 3D Scenes from Human Activitiescited by 22
- 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
- 2024Physics-based Scene Layout Generation from Human Motioncited by 10
- 2026HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactionscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.