# DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama

arXiv: https://arxiv.org/abs/1811.11977 · HTML: https://mlsanity.com/paper/1811.11977/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2018.

## Built on by (most cited first, 30 of 68)

- [LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering](https://mlsanity.com/paper/2104.00568/) (2021)
- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimation](https://mlsanity.com/paper/2304.07803/) (2023)
- [Planar Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2103.14644/) (2021)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [HRDFuse: Monocular 360degDepth Estimation by Collaboratively Learning Holistic-with-Regional Depth Distributions](https://mlsanity.com/paper/2303.11616/) (2023)
- [BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimation](https://mlsanity.com/paper/2209.02952/) (2022)
- [DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization](https://mlsanity.com/paper/2108.10743/) (2021)
- [LGT-Net: Indoor Panoramic Room Layout Estimation with Geometry-Aware Transformer Network](https://mlsanity.com/paper/2203.01824/) (2022)
- [BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learning](https://mlsanity.com/paper/2112.06179/) (2021)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360^circ Radiance Fields](https://mlsanity.com/paper/2208.02705/) (2022)
- [PanelNet: Understanding 360 Indoor Environment via Panel Representation](https://mlsanity.com/paper/2305.09078/) (2023)
- [PanoContext-Former: Panoramic Total Scene Understanding with a Transformer](https://mlsanity.com/paper/2305.12497/) (2023)
- [Plan2Scene: Converting Floorplans to 3D Scenes](https://mlsanity.com/paper/2106.05375/) (2021)
- [Layout-Guided Novel View Synthesis from a Single Indoor Panorama](https://mlsanity.com/paper/2103.17022/) (2021)
- [360 Depth Estimation in the Wild -- The Depth360 Dataset and the SegFuse Network](https://mlsanity.com/paper/2202.08010/) (2022)
- [Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness](https://mlsanity.com/paper/2303.00971/) (2023)
- [PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes](https://mlsanity.com/paper/2106.00446/) (2021)
- [PSMNet: Position-aware Stereo Merging Network for Room Layout Estimation](https://mlsanity.com/paper/2203.15965/) (2022)
- [3D Room Layout Estimation from a Cubemap of Panorama Image via Deep Manhattan Hough Transform](https://mlsanity.com/paper/2207.09291/) (2022)
- [High-Resolution Depth Estimation for 360-degree Panoramas through Perspective and Panoramic Depth Images Registration](https://mlsanity.com/paper/2210.10414/) (2022)
- [3D Scene Geometry Estimation from 360^circ Imagery: A Survey](https://mlsanity.com/paper/2401.09252/) (2024)
- [GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network](https://mlsanity.com/paper/2210.11419/) (2022)
- [SSLayout360: Semi-Supervised Indoor Layout Estimation from 360-Degree Panorama](https://mlsanity.com/paper/2103.13696/) (2021)
- [OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas](https://mlsanity.com/paper/2104.09403/) (2021)
- [Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations](https://mlsanity.com/paper/2301.02184/) (2023)
- [Single-Shot Cuboids: Geodesics-based End-to-end Manhattan Aligned Layout Estimation from Spherical Panoramas](https://mlsanity.com/paper/2102.03939/) (2021)
- [Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image](https://mlsanity.com/paper/2104.07986/) (2021)
- [Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects](https://mlsanity.com/paper/2205.02714/) (2022)

## Abstract (arXiv)

We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama. To achieve better prediction accuracy, our method leverages two projections of the panorama at once, namely the equirectangular panorama-view and the perspective ceiling-view, that each contains different clues about the room layouts. Our network architecture consists of two encoder-decoder branches for analyzing each of the two views. In addition, a novel feature fusion structure is proposed to connect the two branches, which are then jointly trained to predict the 2D floor plans and layout heights. To learn more complex room layouts, we introduce the Realtor360 dataset that contains panoramas of Manhattan-world room layouts with different numbers of corners. Experimental results show that our work outperforms recent state-of-the-art in prediction accuracy and performance, especially in the rooms with non-cuboid layouts.