# A Generalized LLM-Augmented BIM Framework: Application to a Speech-to-BIM system

arXiv: https://arxiv.org/abs/2409.18345 · HTML: https://mlsanity.com/paper/2409.18345/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 15 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 2 of 2)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Interactive Design by Integrating a Large Pre-Trained Language Model and Building Information Modeling](https://mlsanity.com/paper/2306.14165/) (2023)

## Built on by (most cited first, 2 of 2)

- [BIM Information Extraction Through LLM-based Adaptive Exploration](https://mlsanity.com/paper/2605.01698/) (2026)
- [Automating construction safety inspections using a multi-modal vision-language RAG framework](https://mlsanity.com/paper/2510.04145/) (2025)

## Abstract (arXiv)

Performing building information modeling (BIM) tasks is a complex process that imposes a steep learning curve and a heavy cognitive load due to the necessity of remembering sequences of numerous commands. With the rapid advancement of large language models (LLMs), it is foreseeable that BIM tasks, including querying and managing BIM data, 4D and 5D BIM, design compliance checking, or authoring a design, using written or spoken natural language (i.e., text-to-BIM or speech-to-BIM), will soon supplant traditional graphical user interfaces. This paper proposes a generalized LLM-augmented BIM framework to expedite the development of LLM-enhanced BIM applications by providing a step-by-step development process. The proposed framework consists of six steps: interpret-fill-match-structure-execute-check. The paper demonstrates the applicability of the proposed framework through implementing a speech-to-BIM application, NADIA-S (Natural-language-based Architectural Detailing through Interaction with Artificial Intelligence via Speech), using exterior wall detailing as an example.