# Sketch2BIM: A Multi-Agent Human-AI Collaborative Pipeline to Convert Hand-Drawn Floor Plans to 3D BIM

arXiv: https://arxiv.org/abs/2510.20838 · HTML: https://mlsanity.com/paper/2510.20838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 51 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 12 of 12)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [MM-LLMs: Recent Advances in MultiModal Large Language Models](https://mlsanity.com/paper/2401.13601/) (2024)
- [Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2311.13165/) (2023)
- [SketchGen: Generating Constrained CAD Sketches](https://mlsanity.com/paper/2106.02711/) (2021)
- [Plan2Scene: Converting Floorplans to 3D Scenes](https://mlsanity.com/paper/2106.05375/) (2021)
- [BIMgent: Towards Autonomous Building Modeling via Computer-use Agents](https://mlsanity.com/paper/2506.07217/) (2025)
- [HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Model](https://mlsanity.com/paper/2406.20077/) (2024)
- [PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervision](https://mlsanity.com/paper/2407.13394/) (2024)
- [Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Framework](https://mlsanity.com/paper/2408.08054/) (2024)
- [Automatic Reconstruction of Semantic 3D Models from 2D Floor Plans](https://mlsanity.com/paper/2306.01642/) (2023)
- [GroundUp: Rapid Sketch-Based 3D City Massing](https://mlsanity.com/paper/2407.12739/) (2024)

## Abstract (arXiv)

This study introduces a human-in-the-loop pipeline that converts unscaled, hand-drawn floor plan sketches into semantically consistent 3D BIM models. The workflow leverages multimodal large language models (MLLMs) within a multi-agent framework, combining perceptual extraction, human feedback, schema validation, and automated BIM scripting. Initially, sketches are iteratively refined into a structured JSON layout of walls, doors, and windows. Later, these layouts are transformed into executable scripts that generate 3D BIM models. Experiments on ten diverse floor plans demonstrate strong convergence: openings (doors, windows) are captured with high reliability in the initial pass, while wall detection begins around 83% and achieves near-perfect alignment after a few feedback iterations. Across all categories, precision, recall, and F1 scores remain above 0.83, and geometric errors (RMSE, MAE) progressively decrease to zero through feedback corrections. This study demonstrates how MLLM-driven multi-agent reasoning can make BIM creation accessible to both experts and non-experts using only freehand sketches.