# Query2CAD: Generating CAD models using natural language queries

arXiv: https://arxiv.org/abs/2406.00144 · HTML: https://mlsanity.com/paper/2406.00144/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Adaptive Input Representations for Neural Language Modeling](https://mlsanity.com/paper/1809.10853/) (2018)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images](https://mlsanity.com/paper/2002.08988/) (2020)
- [MeshNet: Mesh Neural Network for 3D Shape Representation](https://mlsanity.com/paper/1811.11424/) (2018)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [Leveraging 2D Data to Learn Textured 3D Mesh Generation](https://mlsanity.com/paper/2004.04180/) (2020)
- [3D Shape Segmentation with Projective Convolutional Networks](https://mlsanity.com/paper/1612.02808/) (2016)
- [Learning single-image 3D reconstruction by generative modelling of shape, pose and shading](https://mlsanity.com/paper/1901.06447/) (2019)
- [3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows](https://mlsanity.com/paper/2210.11603/) (2022)
- [Think about it! Improving defeasible reasoning by first modeling the question scenario](https://mlsanity.com/paper/2110.12349/) (2021)
- [Lifelong Robot Learning with Human Assisted Language Planners](https://mlsanity.com/paper/2309.14321/) (2023)
- [Conceptual Design Generation Using Large Language Models](https://mlsanity.com/paper/2306.01779/) (2023)
- [Sketch2Prototype: Rapid Conceptual Design Exploration and Prototyping with Generative AI](https://mlsanity.com/paper/2405.12985/) (2024)
- [A Probe Towards Understanding GAN and VAE Models](https://mlsanity.com/paper/1812.05676/) (2018)
- [IC3D: Image-Conditioned 3D Diffusion for Shape Generation](https://mlsanity.com/paper/2211.10865/) (2022)
- [Addressing GAN Training Instabilities via Tunable Classification Losses](https://mlsanity.com/paper/2310.18291/) (2023)

## Built on by (most cited first, 29 of 29)

- [CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM](https://mlsanity.com/paper/2411.04954/) (2024)
- [CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation](https://mlsanity.com/paper/2505.04481/) (2025)
- [CAD-Recode: Reverse Engineering CAD Code from Point Clouds](https://mlsanity.com/paper/2412.14042/) (2024)
- [Text-to-CadQuery: A New Paradigm for CAD Generation with Scalable Large Model Capabilities](https://mlsanity.com/paper/2505.06507/) (2025)
- [CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers](https://mlsanity.com/paper/2412.13810/) (2024)
- [Large Language Models for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2505.08137/) (2025)
- [cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning](https://mlsanity.com/paper/2505.22914/) (2025)
- [Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Reset](https://mlsanity.com/paper/2409.17126/) (2024)
- [CADSmith: Multi-Agent CAD Generation with Programmatic Geometric Validation](https://mlsanity.com/paper/2603.26512/) (2026)
- [Text-to-CAD Evaluation with CADTests](https://mlsanity.com/paper/2605.07807/) (2026)
- [Explain Like I'm Five: Using LLMs to Improve PDE Surrogate Models with Text](https://mlsanity.com/paper/2410.01137/) (2024)
- [Clarify Before You Draw: Proactive Agents for Robust Text-to-CAD Generation](https://mlsanity.com/paper/2602.03045/) (2026)
- [OptMetaOpenFOAM: Large Language Model Driven Chain of Thought for Sensitivity Analysis and Parameter Optimization based on CFD](https://mlsanity.com/paper/2503.01273/) (2025)
- [CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation](https://mlsanity.com/paper/2504.20830/) (2025)
- [CADReasoner: Iterative Program Editing for CAD Reverse Engineering](https://mlsanity.com/paper/2603.29847/) (2026)
- [Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence](https://mlsanity.com/paper/2606.15932/) (2026)
- [A Survey of AI Methods for Geometry Preparation and Mesh Generation in Engineering Simulation](https://mlsanity.com/paper/2512.23719/) (2025)
- [neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing](https://mlsanity.com/paper/2604.16170/) (2026)
- [UniCAD: A Unified Benchmark and Universal Model for Multi-Modal Multi-Task CAD](https://mlsanity.com/paper/2606.05058/) (2026)
- [Large Language Model-Based Intelligent Antenna Design System](https://mlsanity.com/paper/2504.18271/) (2025)
- [BrepLLM: Enabling Large Language Models to Understand Boundary Representations](https://mlsanity.com/paper/2512.16413/) (2025)
- [STEP-LLM: Generating CAD STEP Models from Natural Language with Large Language Models](https://mlsanity.com/paper/2601.12641/) (2026)
- [PLLM: Pseudo-Labeling Large Language Models for CAD Program Synthesis](https://mlsanity.com/paper/2602.12561/) (2026)
- [Learning Hierarchical and Geometry-Aware Graph Representations for Text-to-CAD](https://mlsanity.com/paper/2604.10075/) (2026)
- [Memory-Augmented Reinforcement Learning Agent for CAD Generation](https://mlsanity.com/paper/2605.19748/) (2026)
- [GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding](https://mlsanity.com/paper/2606.07024/) (2026)
- [Enhancing Creativity in 3D Generative Design via a TRIZ-Inspired Text-to-CAD Framework](https://mlsanity.com/paper/2606.21378/) (2026)
- [PCBWorld: A Benchmark Environment for Engine-Grounded PCB Design Automation](https://mlsanity.com/paper/2607.05915/) (2026)
- [Test-Time Scaling for CAD Generation via Verifier-Free Consensus Selection](https://mlsanity.com/paper/2608.09706/) (2026)

## Abstract (arXiv)

Computer Aided Design (CAD) engineers typically do not achieve their best prototypes in a single attempt. Instead, they iterate and refine their designs to achieve an optimal solution through multiple revisions. This traditional approach, though effective, is time-consuming and relies heavily on the expertise of skilled engineers. To address these challenges, we introduce Query2CAD, a novel framework to generate CAD designs. The framework uses a large language model to generate executable CAD macros. Additionally, Query2CAD refines the generation of the CAD model with the help of its self-refinement loops. Query2CAD operates without supervised data or additional training, using the LLM as both a generator and a refiner. The refiner leverages feedback generated by the BLIP2 model, and to address false negatives, we have incorporated human-in-the-loop feedback into our system. Additionally, we have developed a dataset that encompasses most operations used in CAD model designing and have evaluated our framework using this dataset. Our findings reveal that when we used GPT-4 Turbo as our language model, the architecture achieved a success rate of 53.6% on the first attempt. With subsequent refinements, the success rate increased by 23.1%. In particular, the most significant improvement in the success rate was observed with the first iteration of the refinement. With subsequent refinements, the accuracy of the correct designs did not improve significantly. We have open-sourced our data, model, and code (github.com/akshay140601/Query2CAD).