# BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling

arXiv: https://arxiv.org/abs/2606.20146 · HTML: https://mlsanity.com/paper/2606.20146/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Builds on (most cited first, 28 of 28)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [ScanQA: 3D Question Answering for Spatial Scene Understanding](https://mlsanity.com/paper/2112.10482/) (2021)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark](https://mlsanity.com/paper/2412.07825/) (2024)
- [CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation](https://mlsanity.com/paper/2505.04481/) (2025)
- [Vitruvion: A Generative Model of Parametric CAD Sketches](https://mlsanity.com/paper/2109.14124/) (2021)
- [CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward](https://mlsanity.com/paper/2505.19713/) (2025)
- [Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Models](https://mlsanity.com/paper/2501.19054/) (2025)
- [On the Effectiveness of Large Language Models in Domain-Specific Code Generation](https://mlsanity.com/paper/2312.01639/) (2023)
- [A Survey on Evaluating Large Language Models in Code Generation Tasks](https://mlsanity.com/paper/2408.16498/) (2024)
- [CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images](https://mlsanity.com/paper/2504.04753/) (2025)
- [BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing](https://mlsanity.com/paper/2504.01786/) (2025)
- [TransCAD: A Hierarchical Transformer for CAD Sequence Inference from Point Clouds](https://mlsanity.com/paper/2407.12702/) (2024)
- [BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement](https://mlsanity.com/paper/2412.14203/) (2024)
- [Can Multimodal Large Language Models Understand Spatial Relations?](https://mlsanity.com/paper/2505.19015/) (2025)
- [CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing](https://mlsanity.com/paper/2502.03997/) (2025)
- [Large Language Models for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2505.08137/) (2025)
- [From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach](https://mlsanity.com/paper/2412.11892/) (2024)
- [BIM-GPT: a Prompt-Based Virtual Assistant Framework for BIM Information Retrieval](https://mlsanity.com/paper/2304.09333/) (2023)
- [BIMgent: Towards Autonomous Building Modeling via Computer-use Agents](https://mlsanity.com/paper/2506.07217/) (2025)
- [QueryCAD: Grounded Question Answering for CAD Models](https://mlsanity.com/paper/2409.08704/) (2024)
- [GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback](https://mlsanity.com/paper/2603.27448/) (2026)
- [How Far Are Vision-Language Models from Constructing the Real World? A Benchmark for Physical Generative Reasoning](https://mlsanity.com/paper/2603.24866/) (2026)
- [Text-to-Code Generation for Modular Building Layouts in Building Information Modeling](https://mlsanity.com/paper/2509.23713/) (2025)

## Built on by (most cited first, 1 of 1)

- [EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?](https://mlsanity.com/paper/2609.37686/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly applied to computer-aided design (CAD) to generate design artifacts from textual instructions. In engineering practice, this requires more than creating new geometry, models must also understand existing scenes, edit them correctly, and preserve semantics and relations. However, many CAD benchmarks focus on creating new models rather than editing existing ones, and mostly evaluate geometric correctness. We introduce BIM-Edit, a benchmark for evaluating LLMs on natural-language editing of Building Information Models (BIM) represented in the Industry Foundation Classes (IFC) format. BIM provides a challenging testbed because building models encode geometry together with semantic and relational structure. BIM-Edit contains 324 editing tasks spanning 11 realistic building models and 36 synthetic scenes. Tasks are expressed using three instruction categories - direct, spatial, and topological - covering both explicit and scene-grounded edits. We evaluate outputs along three dimensions: geometric accuracy, semantic validity, and topological consistency. Across evaluated LLMs, the best-performing model achieves only 49.5% average score across the three metrics, and no model fully solves more than 3.4% of tasks. These results demonstrate a substantial gap between current LLM capabilities and the requirements of structured engineering design workflows.