# WAFFLE: Multimodal Floorplan Understanding in the Wild

arXiv: https://arxiv.org/abs/2412.00955 · HTML: https://mlsanity.com/paper/2412.00955/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [MineGAN: effective knowledge transfer from GANs to target domains with few images](https://mlsanity.com/paper/1912.05270/) (2019)
- [Image Generation From Small Datasets via Batch Statistics Adaptation](https://mlsanity.com/paper/1904.01774/) (2019)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [READ: Recursive Autoencoders for Document Layout Generation](https://mlsanity.com/paper/1909.00302/) (2019)
- [CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis](https://mlsanity.com/paper/1904.01920/) (2019)
- [Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries](https://mlsanity.com/paper/2211.15658/) (2022)
- [FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans](https://mlsanity.com/paper/1804.00090/) (2018)
- [Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation](https://mlsanity.com/paper/2007.09841/) (2020)
- [FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spotting](https://mlsanity.com/paper/2105.07147/) (2021)
- [Deep Floor Plan Recognition Using a Multi-Task Network with Room-Boundary-Guided Attention](https://mlsanity.com/paper/1908.11025/) (2019)
- [Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision](https://mlsanity.com/paper/2108.05863/) (2021)
- [Plan2Scene: Converting Floorplans to 3D Scenes](https://mlsanity.com/paper/2106.05375/) (2021)
- [PuzzleFusion: Unleashing the Power of Diffusion Models for Spatial Puzzle Solving](https://mlsanity.com/paper/2211.13785/) (2022)
- [Graph2Plan: Learning Floorplan Generation from Layout Graphs](https://mlsanity.com/paper/2004.13204/) (2020)
- [What's in a Decade? Transforming Faces Through Time](https://mlsanity.com/paper/2210.06642/) (2022)
- [LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models](https://mlsanity.com/paper/2309.00789/) (2023)

## Built on by (most cited first, 5 of 5)

- [C3Po: Cross-View Cross-Modality Correspondence by Pointmap Prediction](https://mlsanity.com/paper/2511.18559/) (2025)
- [CAGE: Continuity-Aware edGE Network Unlocks Robust Floorplan Reconstruction](https://mlsanity.com/paper/2509.15459/) (2025)
- [LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People](https://mlsanity.com/paper/2604.23970/) (2026)
- [When Should a Network Emit Geometry, and When Should It Detect It? Readout, Reconciliation, and Representation in Floorplan Vectorization](https://mlsanity.com/paper/2608.25608/) (2026)
- [ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark](https://mlsanity.com/paper/2609.34047/) (2026)

## Abstract (arXiv)

Buildings are a central feature of human culture and are increasingly being analyzed with computational methods. However, recent works on computational building understanding have largely focused on natural imagery of buildings, neglecting the fundamental element defining a building's structure -- its floorplan. Conversely, existing works on floorplan understanding are extremely limited in scope, often focusing on floorplans of a single semantic category and region (e.g. floorplans of apartments from a single country). In this work, we introduce WAFFLE, a novel multimodal floorplan understanding dataset of nearly 20K floorplan images and metadata curated from Internet data spanning diverse building types, locations, and data formats. By using a large language model and multimodal foundation models, we curate and extract semantic information from these images and their accompanying noisy metadata. We show that WAFFLE enables progress on new building understanding tasks, both discriminative and generative, which were not feasible using prior datasets. We will publicly release WAFFLE along with our code and trained models, providing the research community with a new foundation for learning the semantics of buildings.