# Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries

arXiv: https://arxiv.org/abs/2211.15658 · HTML: https://mlsanity.com/paper/2211.15658/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [PolyTransform: Deep Polygon Transformer for Instance Segmentation](https://mlsanity.com/paper/1912.02801/) (2019)
- [Line Segment Detection Using Transformers without Edges](https://mlsanity.com/paper/2101.01909/) (2021)
- [Text Spotting Transformers](https://mlsanity.com/paper/2204.01918/) (2022)
- [Holistically-Attracted Wireframe Parsing](https://mlsanity.com/paper/2003.01663/) (2020)
- [SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans](https://mlsanity.com/paper/2003.12622/) (2020)
- [Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path](https://mlsanity.com/paper/1908.06702/) (2019)
- [FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans](https://mlsanity.com/paper/1804.00090/) (2018)
- [PlaneTR: Structure-Guided Transformers for 3D Plane Recovery](https://mlsanity.com/paper/2107.13108/) (2021)
- [HEAT: Holistic Edge Attention Transformer for Structured Reconstruction](https://mlsanity.com/paper/2111.15143/) (2021)
- [MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans](https://mlsanity.com/paper/2103.11161/) (2021)
- [Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans](https://mlsanity.com/paper/1812.06677/) (2018)

## Built on by (most cited first, 30 of 49)

- [OpenMask3D: Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2306.13631/) (2023)
- [SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model](https://mlsanity.com/paper/2403.13064/) (2024)
- [OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views](https://mlsanity.com/paper/2404.03650/) (2024)
- [SpatialLM: Training Large Language Models for Structured Indoor Modeling](https://mlsanity.com/paper/2506.07491/) (2025)
- [PolyDiffuse: Polygonal Shape Reconstruction via Guided Set Diffusion Models](https://mlsanity.com/paper/2306.01461/) (2023)
- [TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning](https://mlsanity.com/paper/2406.15658/) (2024)
- [P2P-Bridge: Diffusion Bridges for 3D Point Cloud Denoising](https://mlsanity.com/paper/2408.16325/) (2024)
- [LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories](https://mlsanity.com/paper/2311.12174/) (2023)
- [PolyRoom: Room-aware Transformer for Floorplan Reconstruction](https://mlsanity.com/paper/2407.10439/) (2024)
- [Generating 3D House Wireframes with Semantics](https://mlsanity.com/paper/2407.12267/) (2024)
- [UnLoc: Leveraging Depth Uncertainties for Floorplan Localization](https://mlsanity.com/paper/2509.11301/) (2025)
- [Beyond the Frontier: Predicting Unseen Walls from Occupancy Grids by Learning from Floor Plans](https://mlsanity.com/paper/2406.09160/) (2024)
- [ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction](https://mlsanity.com/paper/2311.18068/) (2023)
- [FRI-Net: Floorplan Reconstruction via Room-wise Implicit Representation](https://mlsanity.com/paper/2407.10687/) (2024)
- [RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings](https://mlsanity.com/paper/2407.14920/) (2024)
- [QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understanding](https://mlsanity.com/paper/2404.06442/) (2024)
- [WAFFLE: Multimodal Floorplan Understanding in the Wild](https://mlsanity.com/paper/2412.00955/) (2024)
- [ScaleLSD: Scalable Deep Line Segment Detection Streamlined](https://mlsanity.com/paper/2506.09369/) (2025)
- [FloorplanVLM: A Vision-Language Model for Floorplan Vectorization](https://mlsanity.com/paper/2602.06507/) (2026)
- [Point2Graph: An End-to-end Point Cloud-based 3D Open-Vocabulary Scene Graph for Robot Navigation](https://mlsanity.com/paper/2409.10350/) (2024)
- [Human-in-the-Loop Local Corrections of 3D Scene Layouts via Infilling](https://mlsanity.com/paper/2503.11806/) (2025)
- [No More Ambiguity in 360deg Room Layout via Bi-Layout Estimation](https://mlsanity.com/paper/2404.09993/) (2024)
- [Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model](https://mlsanity.com/paper/2502.16779/) (2025)
- [CAGE: Continuity-Aware edGE Network Unlocks Robust Floorplan Reconstruction](https://mlsanity.com/paper/2509.15459/) (2025)
- [Historical Astronomical Diagrams Decomposition in Geometric Primitives](https://mlsanity.com/paper/2403.08721/) (2024)
- [BADGR: Bundle Adjustment Diffusion Conditioned by GRadients for Wide-Baseline Floor Plan Reconstruction](https://mlsanity.com/paper/2503.19340/) (2025)
- [Open-Vocabulary Indoor Object Grounding with 3D Hierarchical Scene Graph](https://mlsanity.com/paper/2507.12123/) (2025)
- [PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment](https://mlsanity.com/paper/2508.04659/) (2025)
- [SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs](https://mlsanity.com/paper/2602.22716/) (2026)
- [FloorSAM: SAM-Guided Floorplan Reconstruction with Semantic-Geometric Fusion](https://mlsanity.com/paper/2509.15750/) (2025)

## Abstract (arXiv)

We address 2D floorplan reconstruction from 3D scans. Existing approaches typically employ heuristically designed multi-stage pipelines. Instead, we formulate floorplan reconstruction as a single-stage structured prediction task: find a variable-size set of polygons, which in turn are variable-length sequences of ordered vertices. To solve it we develop a novel Transformer architecture that generates polygons of multiple rooms in parallel, in a holistic manner without hand-crafted intermediate stages. The model features two-level queries for polygons and corners, and includes polygon matching to make the network end-to-end trainable. Our method achieves a new state-of-the-art for two challenging datasets, Structured3D and SceneCAD, along with significantly faster inference than previous methods. Moreover, it can readily be extended to predict additional information, i.e., semantic room types and architectural elements like doors and windows. Our code and models are available at: https://github.com/ywyue/RoomFormer.