# Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Framework

arXiv: https://arxiv.org/abs/2408.08054 · HTML: https://mlsanity.com/paper/2408.08054/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 68 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code](https://mlsanity.com/paper/2403.01248/) (2024)
- [3D-GPT: Procedural 3D Modeling with Large Language Models](https://mlsanity.com/paper/2310.12945/) (2023)
- [I-Design: Personalized LLM Interior Designer](https://mlsanity.com/paper/2404.02838/) (2024)
- [HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising](https://mlsanity.com/paper/2211.13287/) (2022)
- [Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback](https://mlsanity.com/paper/2304.10750/) (2023)
- [ActFloor-GAN: Activity-Guided Adversarial Networks for Human-Centric Floorplan Design](https://mlsanity.com/paper/2111.03545/) (2021)
- [Generative AI Models for Different Steps in Architectural Design: A Literature Review](https://mlsanity.com/paper/2404.01335/) (2024)
- [Generative AIBIM: An automatic and intelligent structural design pipeline integrating BIM and generative AI](https://mlsanity.com/paper/2311.04052/) (2023)
- [Towards a copilot in BIM authoring tool using a large language model-based agent for intelligent human-machine interaction](https://mlsanity.com/paper/2406.16903/) (2024)
- [Vitruvio: 3D Building Meshes via Single Perspective Sketches](https://mlsanity.com/paper/2210.13634/) (2022)
- [Sketch-to-Architecture: Generative AI-aided Architectural Design](https://mlsanity.com/paper/2403.20186/) (2024)
- [Towards commands recommender system in BIM authoring tool using transformers](https://mlsanity.com/paper/2406.10237/) (2024)

## Built on by (most cited first, 9 of 9)

- [BIMgent: Towards Autonomous Building Modeling via Computer-use Agents](https://mlsanity.com/paper/2506.07217/) (2025)
- [Automating Structural Engineering Workflows with Large Language Model Agents](https://mlsanity.com/paper/2510.11004/) (2025)
- [MCP4IFC: IFC-Based Building Design Using Large Language Models](https://mlsanity.com/paper/2511.05533/) (2025)
- [A Modular Reference Architecture for MCP-Servers Enabling Agentic BIM Interaction](https://mlsanity.com/paper/2601.00809/) (2025)
- [Sketch2BIM: A Multi-Agent Human-AI Collaborative Pipeline to Convert Hand-Drawn Floor Plans to 3D BIM](https://mlsanity.com/paper/2510.20838/) (2025)
- [TLC-Plan: A Two-Level Codebook Based Network for End-to-End Vector Floorplan Generation](https://mlsanity.com/paper/2602.07100/) (2026)
- [Multi-Agentic Approach for History Matching of Oil Reservoirs](https://mlsanity.com/paper/2605.15028/) (2026)
- [A Building as a Repository: KIR, a Typed Intermediate Representation for Agent-Authored Building Information Models](https://mlsanity.com/paper/2609.14578/) (2026)
- [K-OPSD: Verifiable On-Policy Self-Distillation for Post-Training Vision-Language Models on AEC Drawings](https://mlsanity.com/paper/2609.34082/) (2026)

## Abstract (arXiv)

The conventional BIM authoring process typically requires designers to master complex and tedious modeling commands in order to materialize their design intentions within BIM authoring tools. This additional cognitive burden complicates the design process and hinders the adoption of BIM and model-based design in the AEC (Architecture, Engineering, and Construction) industry. To facilitate the expression of design intentions more intuitively, we propose Text2BIM, an LLM-based multi-agent framework that can generate 3D building models from natural language instructions. This framework orchestrates multiple LLM agents to collaborate and reason, transforming textual user input into imperative code that invokes the BIM authoring tool's APIs, thereby generating editable BIM models with internal layouts, external envelopes, and semantic information directly in the software. Furthermore, a rule-based model checker is introduced into the agentic workflow, utilizing predefined domain knowledge to guide the LLM agents in resolving issues within the generated models and iteratively improving model quality. Extensive experiments were conducted to compare and analyze the performance of three different LLMs under the proposed framework. The evaluation results demonstrate that our approach can effectively generate high-quality, structurally rational building models that are aligned with the abstract concepts specified by user input. Finally, an interactive software prototype was developed to integrate the framework into the BIM authoring software Vectorworks, showcasing the potential of modeling by chatting. The code is available at: https://github.com/dcy0577/Text2BIM