# CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing

arXiv: https://arxiv.org/abs/2502.03997 · HTML: https://mlsanity.com/paper/2502.03997/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Self-improving MLLMs and synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-improving MLLMs and synthetic data](https://mlsanity.com/synth/self-improving-mllms-and-synthetic-data/)

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [Pix2Video: Video Editing using Image Diffusion](https://mlsanity.com/paper/2303.12688/) (2023)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [StyleDrop: Text-to-Image Generation in Any Style](https://mlsanity.com/paper/2306.00983/) (2023)
- [StableVideo: Text-driven Consistency-aware Diffusion Video Editing](https://mlsanity.com/paper/2308.09592/) (2023)
- [MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators](https://mlsanity.com/paper/2306.10900/) (2023)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [Fine-Tuned Language Models Generate Stable Inorganic Materials as Text](https://mlsanity.com/paper/2402.04379/) (2024)
- [Hierarchical Neural Coding for Controllable CAD Model Generation](https://mlsanity.com/paper/2307.00149/) (2023)
- [SKED: Sketch-guided Text-based 3D Editing](https://mlsanity.com/paper/2303.10735/) (2023)
- [CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention](https://mlsanity.com/paper/2402.17678/) (2024)
- [How Can Large Language Models Help Humans in Design and Manufacturing?](https://mlsanity.com/paper/2307.14377/) (2023)
- [Event Grounded Criminal Court View Generation with Cooperative (Large) Language Models](https://mlsanity.com/paper/2404.07001/) (2024)

## Built on by (most cited first, 19 of 19)

- [Large Language Models for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2505.08137/) (2025)
- [AutoBrep: Autoregressive B-Rep Generation with Unified Topology and Geometry](https://mlsanity.com/paper/2512.03018/) (2025)
- [cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning](https://mlsanity.com/paper/2505.22914/) (2025)
- [BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD](https://mlsanity.com/paper/2605.10865/) (2026)
- [ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models](https://mlsanity.com/paper/2512.06328/) (2025)
- [Text-to-CAD Evaluation with CADTests](https://mlsanity.com/paper/2605.07807/) (2026)
- [IterCAD: An Iterative Multimodal Agent for Visually-Grounded CAD Generation and Editing](https://mlsanity.com/paper/2606.13368/) (2026)
- [CADReasoner: Iterative Program Editing for CAD Reverse Engineering](https://mlsanity.com/paper/2603.29847/) (2026)
- [GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models](https://mlsanity.com/paper/2506.10337/) (2025)
- [Simulating Human-Like Learning Dynamics with LLM-Empowered Agents](https://mlsanity.com/paper/2508.05622/) (2025)
- [neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing](https://mlsanity.com/paper/2604.16170/) (2026)
- [PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models](https://mlsanity.com/paper/2604.19773/) (2026)
- [CAD-Tokenizer: Towards Text-based CAD Prototyping via Modality-Specific Tokenization](https://mlsanity.com/paper/2509.21150/) (2025)
- [BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling](https://mlsanity.com/paper/2606.20146/) (2026)
- [CADENA: Stepwise CAD Reverse Engineering](https://mlsanity.com/paper/2608.00799/) (2026)
- [CADMorph: Geometry-Driven Parametric CAD Editing via a Plan-Generate-Verify Loop](https://mlsanity.com/paper/2512.11480/) (2025)
- [ReLayout: Versatile and Structure-Preserving Design Layout Editing via Relation-Aware Design Reconstruction](https://mlsanity.com/paper/2602.01046/) (2026)
- [KDH-CAD: Knowledge-data hybrid CAD learning under data scarcity](https://mlsanity.com/paper/2606.01702/) (2026)
- [AgenticCADedit: A Stateful, Tool-Mediated Agentic Approach to Multimodal 3D CAD Editing](https://mlsanity.com/paper/2609.29621/) (2026)

## Abstract (arXiv)

Computer Aided Design (CAD) is indispensable across various industries. Text-based CAD editing, which automates the modification of CAD models based on textual instructions, holds great potential but remains underexplored. Existing methods primarily focus on design variation generation or text-based CAD generation, either lacking support for text-based control or neglecting existing CAD models as constraints. We introduce CAD-Editor, the first framework for text-based CAD editing. To address the challenge of demanding triplet data with accurate correspondence for training, we propose an automated data synthesis pipeline. This pipeline utilizes design variation models to generate pairs of original and edited CAD models and employs Large Vision-Language Models (LVLMs) to summarize their differences into editing instructions. To tackle the composite nature of text-based CAD editing, we propose a locate-then-infill framework that decomposes the task into two focused sub-tasks: locating regions requiring modification and infilling these regions with appropriate edits. Large Language Models (LLMs) serve as the backbone for both sub-tasks, leveraging their capabilities in natural language understanding and CAD knowledge. Experiments show that CAD-Editor achieves superior performance both quantitatively and qualitatively. The code is available at url https://github.com/microsoft/CAD-Editor.