# Layout-Guided Novel View Synthesis from a Single Indoor Panorama

arXiv: https://arxiv.org/abs/2103.17022 · HTML: https://mlsanity.com/paper/2103.17022/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 58 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [StructureFlow: Image Inpainting via Structure-aware Appearance Flow](https://mlsanity.com/paper/1908.03852/) (2019)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [Coherent Semantic Attention for Image Inpainting](https://mlsanity.com/paper/1905.12384/) (2019)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [Extreme View Synthesis](https://mlsanity.com/paper/1812.04777/) (2018)
- [Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting](https://mlsanity.com/paper/1904.07475/) (2019)
- [Foreground-aware Image Inpainting](https://mlsanity.com/paper/1901.05945/) (2019)
- [Shift-Net: Image Inpainting via Deep Feature Rearrangement](https://mlsanity.com/paper/1801.09392/) (2018)
- [Layer-structured 3D Scene Inference via View Synthesis](https://mlsanity.com/paper/1807.10264/) (2018)
- [Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](https://mlsanity.com/paper/1704.04086/) (2017)

## Built on by (most cited first, 16 of 16)

- [Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models](https://mlsanity.com/paper/2303.11989/) (2023)
- [PERF: Panoramic Neural Radiance Field from a Single Panorama](https://mlsanity.com/paper/2310.16831/) (2023)
- [360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming](https://mlsanity.com/paper/2402.00763/) (2024)
- [360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360^circ Radiance Fields](https://mlsanity.com/paper/2208.02705/) (2022)
- [Pano2Room: Novel View Synthesis from a Single Indoor Panorama](https://mlsanity.com/paper/2408.11413/) (2024)
- [Taming Stable Diffusion for Text to 360deg Panorama Image Generation](https://mlsanity.com/paper/2404.07949/) (2024)
- [Casual Indoor HDR Radiance Capture from Omnidirectional Images](https://mlsanity.com/paper/2208.07903/) (2022)
- [High-Resolution Depth Estimation for 360-degree Panoramas through Perspective and Panoramic Depth Images Registration](https://mlsanity.com/paper/2210.10414/) (2022)
- [360deg Optical Flow using Tangent Images](https://mlsanity.com/paper/2112.14331/) (2021)
- [HORIZON: High-Resolution Semantically Controlled Panorama Synthesis](https://mlsanity.com/paper/2210.04522/) (2022)
- [Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion](https://mlsanity.com/paper/2509.00843/) (2025)
- [Seg2Reg: Differentiable 2D Segmentation to 1D Regression Rendering for 360 Room Layout Reconstruction](https://mlsanity.com/paper/2311.18695/) (2023)
- [You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting](https://mlsanity.com/paper/2504.09062/) (2025)
- [WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning](https://mlsanity.com/paper/2305.13901/) (2023)
- [Enabling Robust, Real-Time Verification of Vision-Based Navigation through View Synthesis](https://mlsanity.com/paper/2507.02993/) (2025)
- [PanoTPS-Net: Panoramic Room Layout Estimation via Thin Plate Spline Transformation](https://mlsanity.com/paper/2510.11992/) (2025)

## Abstract (arXiv)

Existing view synthesis methods mainly focus on the perspective images and have shown promising results. However, due to the limited field-of-view of the pinhole camera, the performance quickly degrades when large camera movements are adopted. In this paper, we make the first attempt to generate novel views from a single indoor panorama and take the large camera translations into consideration. To tackle this challenging problem, we first use Convolutional Neural Networks (CNNs) to extract the deep features and estimate the depth map from the source-view image. Then, we leverage the room layout prior, a strong structural constraint of the indoor scene, to guide the generation of target views. More concretely, we estimate the room layout in the source view and transform it into the target viewpoint as guidance. Meanwhile, we also constrain the room layout of the generated target-view images to enforce geometric consistency. To validate the effectiveness of our method, we further build a large-scale photo-realistic dataset containing both small and large camera translations. The experimental results on our challenging dataset demonstrate that our method achieves state-of-the-art performance. The project page is at https://github.com/bluestyle97/PNVS.