# Plan2Scene: Converting Floorplans to 3D Scenes

arXiv: https://arxiv.org/abs/2106.05375 · HTML: https://mlsanity.com/paper/2106.05375/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [IM2CAD](https://mlsanity.com/paper/1608.05137/) (2016)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image](https://mlsanity.com/paper/1804.05790/) (2018)
- [Diversified Texture Synthesis with Feed-forward Networks](https://mlsanity.com/paper/1703.01664/) (2017)
- [Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding](https://mlsanity.com/paper/1902.09777/) (2019)
- [Learning a Neural 3D Texture Space from 2D Exemplars](https://mlsanity.com/paper/1912.04158/) (2019)
- [Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks](https://mlsanity.com/paper/1809.00886/) (2018)
- [A Dataset of Multi-Illumination Images in the Wild](https://mlsanity.com/paper/1910.08131/) (2019)
- [GeoLayout: Geometry Driven Room Layout Estimation Based on Depth Maps of Planes](https://mlsanity.com/paper/2008.06286/) (2020)
- [Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans](https://mlsanity.com/paper/1812.06677/) (2018)
- [Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only](https://mlsanity.com/paper/2003.00397/) (2020)
- [Deep Multi-Modal Image Correspondence Learning](https://mlsanity.com/paper/1612.01225/) (2016)

## Built on by (most cited first, 17 of 17)

- [CC3D: Layout-Conditioned Generation of Compositional 3D Scenes](https://mlsanity.com/paper/2303.12074/) (2023)
- [RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture](https://mlsanity.com/paper/2305.11337/) (2023)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Model](https://mlsanity.com/paper/2406.20077/) (2024)
- [TexTile: A Differentiable Metric for Texture Tileability](https://mlsanity.com/paper/2403.12961/) (2024)
- [From Programs to Poses: Factored Real-World Scene Generation via Learned Program Libraries](https://mlsanity.com/paper/2510.10292/) (2025)
- [WAFFLE: Multimodal Floorplan Understanding in the Wild](https://mlsanity.com/paper/2412.00955/) (2024)
- [C3Po: Cross-View Cross-Modality Correspondence by Pointmap Prediction](https://mlsanity.com/paper/2511.18559/) (2025)
- [PanoWorld: A Generative Spatial World Model for Consistent Whole-House Panorama Synthesis](https://mlsanity.com/paper/2605.17916/) (2026)
- [Floorplan-Aware Camera Poses Refinement](https://mlsanity.com/paper/2210.04572/) (2022)
- [MaGRITTe: Manipulative and Generative 3D Realization from Image, Topview and Text](https://mlsanity.com/paper/2404.00345/) (2024)
- [DoorDet: Semi-Automated Multi-Class Door Detection Dataset via Object Detection and Large Language Models](https://mlsanity.com/paper/2508.07714/) (2025)
- [Recognizing and Reconstructing a Multi-Unit Floor Plan](https://mlsanity.com/paper/2408.01526/) (2024)
- [Scene Co-pilot: Procedural Text to Video Generation with Human in the Loop](https://mlsanity.com/paper/2411.18644/) (2024)
- [Top2Pano: Learning to Generate Indoor Panoramas from Top-Down View](https://mlsanity.com/paper/2507.21371/) (2025)
- [Sketch2BIM: A Multi-Agent Human-AI Collaborative Pipeline to Convert Hand-Drawn Floor Plans to 3D BIM](https://mlsanity.com/paper/2510.20838/) (2025)
- [SG-Layout: Structured Scene Graph-Guided Layout Generation with LLMs](https://mlsanity.com/paper/2608.01106/) (2026)

## Abstract (arXiv)

We address the task of converting a floorplan and a set of associated photos of a residence into a textured 3D mesh model, a task which we call Plan2Scene. Our system 1) lifts a floorplan image to a 3D mesh model; 2) synthesizes surface textures based on the input photos; and 3) infers textures for unobserved surfaces using a graph neural network architecture. To train and evaluate our system we create indoor surface texture datasets, and augment a dataset of floorplans and photos from prior work with rectified surface crops and additional annotations. Our approach handles the challenge of producing tileable textures for dominant surfaces such as floors, walls, and ceilings from a sparse set of unaligned photos that only partially cover the residence. Qualitative and quantitative evaluations show that our system produces realistic 3D interior models, outperforming baseline approaches on a suite of texture quality metrics and as measured by a holistic user study.