# Self-supervised 360circ Room Layout Estimation

arXiv: https://arxiv.org/abs/2203.16057 · HTML: https://mlsanity.com/paper/2203.16057/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features](https://mlsanity.com/paper/2011.11498/) (2020)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.03650/) (2016)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Corners for Layout: End-to-End Layout Recovery from 360 Images](https://mlsanity.com/paper/1903.08094/) (2019)
- [LEGO: Learning Edge with Geometry all at Once by Watching Videos](https://mlsanity.com/paper/1803.05648/) (2018)
- [LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering](https://mlsanity.com/paper/2104.00568/) (2021)
- [DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization](https://mlsanity.com/paper/2108.10743/) (2021)
- [SSLayout360: Semi-Supervised Indoor Layout Estimation from 360-Degree Panorama](https://mlsanity.com/paper/2103.13696/) (2021)
- [LayoutMP3D: Layout Annotation of Matterport3D](https://mlsanity.com/paper/2003.13516/) (2020)

## Built on by (most cited first, 2 of 2)

- [Calibrating Panoramic Depth Estimation for Practical Localization and Mapping](https://mlsanity.com/paper/2308.14005/) (2023)
- [MultiDepth: Multi-Sample Priors for Refining Monocular Metric Depth Estimations in Indoor Scenes](https://mlsanity.com/paper/2411.01048/) (2024)

## Abstract (arXiv)

We present the first self-supervised method to train panoramic room layout estimation models without any labeled data. Unlike per-pixel dense depth that provides abundant correspondence constraints, layout representation is sparse and topological, hindering the use of self-supervised reprojection consistency on images. To address this issue, we propose Differentiable Layout View Rendering, which can warp a source image to the target camera pose given the estimated layout from the target image. As each rendered pixel is differentiable with respect to the estimated layout, we can now train the layout estimation model by minimizing reprojection loss. Besides, we introduce regularization losses to encourage Manhattan alignment, ceiling-floor alignment, cycle consistency, and layout stretch consistency, which further improve our predictions. Finally, we present the first self-supervised results on ZilloIndoor and MatterportLayout datasets. Our approach also shows promising solutions in data-scarce scenarios and active learning, which would have an immediate value in the real estate virtual tour software. Code is available at https://github.com/joshua049/Stereo-360-Layout.