# MVLayoutNet:3D layout reconstruction with multi-view panoramas

arXiv: https://arxiv.org/abs/2112.06133 · HTML: https://mlsanity.com/paper/2112.06133/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 12 Dec 2021.

## Builds on (most cited first, 17 of 17)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features](https://mlsanity.com/paper/2011.11498/) (2020)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [Supervised Fitting of Geometric Primitives to 3D Point Clouds](https://mlsanity.com/paper/1811.08988/) (2018)
- [IM2CAD](https://mlsanity.com/paper/1608.05137/) (2016)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Pyramid Multi-view Stereo Net with Self-adaptive View Aggregation](https://mlsanity.com/paper/1912.03001/) (2019)
- [LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering](https://mlsanity.com/paper/2104.00568/) (2021)
- [RoomNet: End-to-End Room Layout Estimation](https://mlsanity.com/paper/1703.06241/) (2017)
- [Pano2CAD: Room Layout From A Single Panorama Image](https://mlsanity.com/paper/1609.09270/) (2016)
- [Joint Layout Estimation and Global Multi-View Registration for Indoor Reconstruction](https://mlsanity.com/paper/1704.07632/) (2017)

## Built on by (most cited first, 6 of 6)

- [CompGS: Efficient 3D Scene Representation via Compressed Gaussian Splatting](https://mlsanity.com/paper/2404.09458/) (2024)
- [AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings](https://mlsanity.com/paper/2406.08960/) (2024)
- [360-MLC: Multi-view Layout Consistency for Self-training and Hyper-parameter Tuning](https://mlsanity.com/paper/2210.12935/) (2022)
- [Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model](https://mlsanity.com/paper/2502.16779/) (2025)
- [PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment](https://mlsanity.com/paper/2508.04659/) (2025)
- [PolyLayout: Multi-room Manhattan Layout Estimation](https://mlsanity.com/paper/2608.03323/) (2026)

## Abstract (arXiv)

We present MVLayoutNet, an end-to-end network for holistic 3D reconstruction from multi-view panoramas. Our core contribution is to seamlessly combine learned monocular layout estimation and multi-view stereo (MVS) for accurate layout reconstruction in both 3D and image space. We jointly train a layout module to produce an initial layout and a novel MVS module to obtain accurate layout geometry. Unlike standard MVSNet [33], our MVS module takes a newly-proposed layout cost volume, which aggregates multi-view costs at the same depth layer into corresponding layout elements. We additionally provide an attention-based scheme that guides the MVS module to focus on structural regions. Such a design considers both local pixel-level costs and global holistic information for better reconstruction. Experiments show that our method outperforms state-of-the-arts in terms of depth rmse by 21.7% and 20.6% on the 2D-3D-S [1] and ZInD [5] datasets. Finally, our method leads to coherent layout geometry that enables the reconstruction of an entire scene.