# Virtual Home Staging: Inverse Rendering and Editing an Indoor Panorama under Natural Illumination

arXiv: https://arxiv.org/abs/2311.12265 · HTML: https://mlsanity.com/paper/2311.12265/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 30 of 31)

- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image](https://mlsanity.com/paper/2002.12212/) (2020)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [IM2CAD](https://mlsanity.com/paper/1608.05137/) (2016)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [Rendering Synthetic Objects into Legacy Photographs](https://mlsanity.com/paper/1912.11565/) (2019)
- [Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1808.02201/) (2018)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videos](https://mlsanity.com/paper/1806.01320/) (2018)
- [DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality](https://mlsanity.com/paper/1904.01175/) (2019)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [Physically-Based Editing of Indoor Scene Lighting from a Single Image](https://mlsanity.com/paper/2205.09343/) (2022)
- [Orientation-aware Semantic Segmentation on Icosahedron Spheres](https://mlsanity.com/paper/1907.12849/) (2019)
- [LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering](https://mlsanity.com/paper/2104.00568/) (2021)
- [Self-Supervised Learning of Depth and Camera Motion from 360deg Videos](https://mlsanity.com/paper/1811.05304/) (2018)
- [Deep Floor Plan Recognition Using a Multi-Task Network with Room-Boundary-Guided Attention](https://mlsanity.com/paper/1908.11025/) (2019)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [What's in my Room? Object Recognition on Indoor Panoramic Images](https://mlsanity.com/paper/1910.06138/) (2019)
- [PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes](https://mlsanity.com/paper/2106.00446/) (2021)
- [Deep Lighting Environment Map Estimation from Spherical Panoramas](https://mlsanity.com/paper/2005.08000/) (2020)
- [No Shadow Left Behind: Removing Objects and their Shadows using Approximate Lighting and Geometry](https://mlsanity.com/paper/2012.10565/) (2020)
- [Towards Full-to-Empty Room Generation with Structure-Aware Feature Encoding and Soft Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/2112.05396/) (2021)
- [Learning Continuous Implicit Representation for Near-Periodic Patterns](https://mlsanity.com/paper/2208.12278/) (2022)
- [Semantically Supervised Appearance Decomposition for Virtual Staging from a Single Panorama](https://mlsanity.com/paper/2205.13150/) (2022)

## Built on by (most cited first, 5 of 5)

- [An Empty Room is All We Want: Automatic Defurnishing of Indoor Panoramas](https://mlsanity.com/paper/2405.03682/) (2024)
- [SceneNAT: Masked Generative Modeling for Language-Guided Indoor Scene Synthesis](https://mlsanity.com/paper/2601.07218/) (2026)
- [Evaluating Panoramic 3D Estimation in Indoor Lighting Analysis](https://mlsanity.com/paper/2403.14836/) (2024)
- [Indoor Heat Estimation from a Single Visible-Light Panorama](https://mlsanity.com/paper/2502.06973/) (2025)
- [Defurnishing with X-Ray Vision: Joint Removal of Furniture from Panoramas and Mesh](https://mlsanity.com/paper/2506.05338/) (2025)

## Abstract (arXiv)

We propose a novel inverse rendering method that enables the transformation of existing indoor panoramas with new indoor furniture layouts under natural illumination. To achieve this, we captured indoor HDR panoramas along with real-time outdoor hemispherical HDR photographs. Indoor and outdoor HDR images were linearly calibrated with measured absolute luminance values for accurate scene relighting. Our method consists of three key components: (1) panoramic furniture detection and removal, (2) automatic floor layout design, and (3) global rendering with scene geometry, new furniture objects, and a real-time outdoor photograph. We demonstrate the effectiveness of our workflow in rendering indoor scenes under different outdoor illumination conditions. Additionally, we contribute a new calibrated HDR (Cali-HDR) dataset that consists of 137 calibrated indoor panoramas and their associated outdoor photographs.