# LGT-Net: Indoor Panoramic Room Layout Estimation with Geometry-Aware Transformer Network

arXiv: https://arxiv.org/abs/2203.01824 · HTML: https://mlsanity.com/paper/2203.01824/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Rethinking Positional Encoding in Language Pre-training](https://mlsanity.com/paper/2006.15595/) (2020)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features](https://mlsanity.com/paper/2011.11498/) (2020)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Corners for Layout: End-to-End Layout Recovery from 360 Images](https://mlsanity.com/paper/1903.08094/) (2019)
- [LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering](https://mlsanity.com/paper/2104.00568/) (2021)
- [Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network](https://mlsanity.com/paper/1907.00939/) (2019)

## Built on by (most cited first, 29 of 29)

- [360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming](https://mlsanity.com/paper/2402.00763/) (2024)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [PanelNet: Understanding 360 Indoor Environment via Panel Representation](https://mlsanity.com/paper/2305.09078/) (2023)
- [Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation](https://mlsanity.com/paper/2510.08673/) (2025)
- [Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness](https://mlsanity.com/paper/2303.00971/) (2023)
- [Cylin-Painting: Seamless 360textdegree Panoramic Image Outpainting and Beyond](https://mlsanity.com/paper/2204.08563/) (2022)
- [3D Room Layout Estimation from a Cubemap of Panorama Image via Deep Manhattan Hough Transform](https://mlsanity.com/paper/2207.09291/) (2022)
- [GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network](https://mlsanity.com/paper/2210.11419/) (2022)
- [PolyRoom: Room-aware Transformer for Floorplan Reconstruction](https://mlsanity.com/paper/2407.10439/) (2024)
- [360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception](https://mlsanity.com/paper/2312.16268/) (2023)
- [Integrating Meshes and 3D Gaussians for Indoor Scene Reconstruction with SAM Mask Guidance](https://mlsanity.com/paper/2407.16173/) (2024)
- [MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold](https://mlsanity.com/paper/2606.13376/) (2026)
- [No More Ambiguity in 360deg Room Layout via Bi-Layout Estimation](https://mlsanity.com/paper/2404.09993/) (2024)
- [Self-training Room Layout Estimation via Geometry-aware Ray-casting](https://mlsanity.com/paper/2407.15041/) (2024)
- [CAGE: Continuity-Aware edGE Network Unlocks Robust Floorplan Reconstruction](https://mlsanity.com/paper/2509.15459/) (2025)
- [U2RLE: Uncertainty-Guided 2-Stage Room Layout Estimation](https://mlsanity.com/paper/2304.08580/) (2023)
- [Shape-Net: Room Layout Estimation from Panoramic Images Robust to Occlusion using Knowledge Distillation with 3D Shapes as Additional Inputs](https://mlsanity.com/paper/2304.12624/) (2023)
- [Seg2Reg: Differentiable 2D Segmentation to 1D Regression Rendering for 360 Room Layout Reconstruction](https://mlsanity.com/paper/2311.18695/) (2023)
- [BADGR: Bundle Adjustment Diffusion Conditioned by GRadients for Wide-Baseline Floor Plan Reconstruction](https://mlsanity.com/paper/2503.19340/) (2025)
- [PanoMixSwap Panorama Mixing via Structural Swapping for Indoor Scene Understanding](https://mlsanity.com/paper/2309.09514/) (2023)
- [TUN3D: Towards Real-World Scene Understanding from Unposed Images](https://mlsanity.com/paper/2509.21388/) (2025)
- [iBARLE: imBalance-Aware Room Layout Estimation](https://mlsanity.com/paper/2308.15050/) (2023)
- [Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier Transform](https://mlsanity.com/paper/2402.18287/) (2024)
- [Semi-Supervised 360 Layout Estimation with Panoramic Collaborative Perturbations](https://mlsanity.com/paper/2503.01114/) (2025)
- [uLayout: Unified Room Layout Estimation for Perspective and Panoramic Images](https://mlsanity.com/paper/2503.21562/) (2025)
- [PanoTPS-Net: Panoramic Room Layout Estimation via Thin Plate Spline Transformation](https://mlsanity.com/paper/2510.11992/) (2025)
- [DreamHome-Pano: Design-Aware and Conflict-Free Panoramic Interior Generation](https://mlsanity.com/paper/2602.06494/) (2026)
- [Gimbal360: Canonicalizing Planar Diffusion for Spherical Panorama Completion](https://mlsanity.com/paper/2603.23179/) (2026)
- [MvFFN: Multi-view Floor plan Feed-forward Network for Unposed Wide-Baseline Panorama Layout Reconstruction](https://mlsanity.com/paper/2610.14315/) (2026)

## Abstract (arXiv)

3D room layout estimation by a single panorama using deep neural networks has made great progress. However, previous approaches can not obtain efficient geometry awareness of room layout with the only latitude of boundaries or horizon-depth. We present that using horizon-depth along with room height can obtain omnidirectional-geometry awareness of room layout in both horizontal and vertical directions. In addition, we propose a planar-geometry aware loss function with normals and gradients of normals to supervise the planeness of walls and turning of corners. We propose an efficient network, LGT-Net, for room layout estimation, which contains a novel Transformer architecture called SWG-Transformer to model geometry relations. SWG-Transformer consists of (Shifted) Window Blocks and Global Blocks to combine the local and global geometry relations. Moreover, we design a novel relative position embedding of Transformer to enhance the spatial identification ability for the panorama. Experiments show that the proposed LGT-Net achieves better performance than current state-of-the-arts (SOTA) on benchmark datasets.