# Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes

arXiv: https://arxiv.org/abs/2207.08656 · HTML: https://mlsanity.com/paper/2207.08656/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 58 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 30 of 49)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image](https://mlsanity.com/paper/2002.12212/) (2020)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [Learning elementary structures for 3D shape generation and matching](https://mlsanity.com/paper/1908.04725/) (2019)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [Holistic 3D Scene Understanding from a Single Image with Implicit Representation](https://mlsanity.com/paper/2103.06422/) (2021)

## Built on by (most cited first, 30 of 44)

- [MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation](https://mlsanity.com/paper/2412.03558/) (2024)
- [CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image](https://mlsanity.com/paper/2502.12894/) (2025)
- [Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View](https://mlsanity.com/paper/2404.03421/) (2024)
- [DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image](https://mlsanity.com/paper/2311.18610/) (2023)
- [PhyRecon: Physically Plausible Neural Scene Reconstruction](https://mlsanity.com/paper/2404.16666/) (2024)
- [Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture](https://mlsanity.com/paper/2311.00457/) (2023)
- [Decompositional Neural Scene Reconstruction with Generative Diffusion Prior](https://mlsanity.com/paper/2503.14830/) (2025)
- [Learning 3D Scene Priors with 2D Supervision](https://mlsanity.com/paper/2211.14157/) (2022)
- [DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion](https://mlsanity.com/paper/2507.22825/) (2025)
- [ShapeR: Robust Conditional 3D Shape Generation from Casual Captures](https://mlsanity.com/paper/2601.11514/) (2026)
- [PanoContext-Former: Panoramic Total Scene Understanding with a Transformer](https://mlsanity.com/paper/2305.12497/) (2023)
- [SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model](https://mlsanity.com/paper/2512.10957/) (2025)
- [Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling](https://mlsanity.com/paper/2411.19492/) (2024)
- [3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image](https://mlsanity.com/paper/2604.04406/) (2026)
- [3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework](https://mlsanity.com/paper/2512.17459/) (2025)
- [Coherent 3D Scene Diffusion From a Single RGB Image](https://mlsanity.com/paper/2412.10294/) (2024)
- [Convex Decomposition of Indoor Scenes](https://mlsanity.com/paper/2307.04246/) (2023)
- [Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wild](https://mlsanity.com/paper/2403.14539/) (2024)
- [Unsupervised Template-assisted Point Cloud Shape Correspondence Network](https://mlsanity.com/paper/2403.16412/) (2024)
- [Reconstructing Animals and the Wild](https://mlsanity.com/paper/2411.18807/) (2024)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence](https://mlsanity.com/paper/2511.21945/) (2025)
- [LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset](https://mlsanity.com/paper/2312.12418/) (2023)
- [Blocks2World: Controlling Realistic Scenes with Editable Primitives](https://mlsanity.com/paper/2307.03847/) (2023)
- [Robust Shape Fitting for 3D Scene Abstraction](https://mlsanity.com/paper/2403.10452/) (2024)
- [M3D: Dual-Stream Selective State Spaces and Depth-Driven Framework for High-Fidelity Single-View 3D Reconstruction](https://mlsanity.com/paper/2411.12635/) (2024)
- [PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstruction](https://mlsanity.com/paper/2603.05888/) (2026)
- [AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environments](https://mlsanity.com/paper/2603.25494/) (2026)
- [Generative Blocks World: Moving Things Around in Pictures](https://mlsanity.com/paper/2506.20703/) (2025)
- [Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image](https://mlsanity.com/paper/2603.05908/) (2026)

## Abstract (arXiv)

We present a new framework to reconstruct holistic 3D indoor scenes including both room background and indoor objects from single-view images. Existing methods can only produce 3D shapes of indoor objects with limited geometry quality because of the heavy occlusion of indoor scenes. To solve this, we propose an instance-aligned implicit function (InstPIFu) for detailed object reconstruction. Combining with instance-aligned attention module, our method is empowered to decouple mixed local features toward the occluded instances. Additionally, unlike previous methods that simply represents the room background as a 3D bounding box, depth map or a set of planes, we recover the fine geometry of the background via implicit representation. Extensive experiments on the SUN RGB-D, Pix3D, 3D-FUTURE, and 3D-FRONT datasets demonstrate that our method outperforms existing approaches in both background and foreground object reconstruction. Our code and model will be made publicly available.