# CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images

arXiv: https://arxiv.org/abs/2504.04753 · HTML: https://mlsanity.com/paper/2504.04753/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers](https://mlsanity.com/paper/2311.15475/) (2023)
- [TripoSR: Fast 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/2403.02151/) (2024)
- [Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions](https://mlsanity.com/paper/2211.13757/) (2022)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry](https://mlsanity.com/paper/2401.15563/) (2024)
- [Hierarchical Neural Coding for Controllable CAD Model Generation](https://mlsanity.com/paper/2307.00149/) (2023)
- [CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention](https://mlsanity.com/paper/2402.17678/) (2024)
- [MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Model](https://mlsanity.com/paper/2408.10198/) (2024)
- [Inferring CAD Modeling Sequences Using Zone Graphs](https://mlsanity.com/paper/2104.03900/) (2021)
- [Engineering Sketch Generation for Computer-Aided Design](https://mlsanity.com/paper/2104.09621/) (2021)
- [SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations](https://mlsanity.com/paper/2303.10613/) (2023)
- [Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry](https://mlsanity.com/paper/2410.03417/) (2024)
- [TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision](https://mlsanity.com/paper/2303.13273/) (2023)

## Built on by (most cited first, 30 of 35)

- [cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning](https://mlsanity.com/paper/2505.22914/) (2025)
- [CReFT-CAD: Boosting Orthographic Projection Reasoning for CAD via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.00568/) (2025)
- [ArtiCAD: Articulated CAD Assembly Design via Multi-Agent Code Generation](https://mlsanity.com/paper/2604.10992/) (2026)
- [ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization](https://mlsanity.com/paper/2505.10250/) (2025)
- [GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models](https://mlsanity.com/paper/2506.10337/) (2025)
- [CADDesigner: Conceptual CAD Model Generation with a General-Purpose Agent](https://mlsanity.com/paper/2508.01031/) (2025)
- [BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning](https://mlsanity.com/paper/2602.22284/) (2026)
- [GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback](https://mlsanity.com/paper/2603.27448/) (2026)
- [neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing](https://mlsanity.com/paper/2604.16170/) (2026)
- [GACO-CAD: Geometry-Augmented and Conciseness-Optimized CAD Model Generation from Single Image](https://mlsanity.com/paper/2510.17157/) (2025)
- [CADKnitter: Compositional CAD Generation from Text and Geometry Guidance](https://mlsanity.com/paper/2512.11199/) (2025)
- [Draw it like Euclid: Teaching transformer models to generate CAD profiles using ruler and compass construction steps](https://mlsanity.com/paper/2601.09428/) (2026)
- [BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling](https://mlsanity.com/paper/2606.20146/) (2026)
- [Pointer-CAD v2: Plan-Then-Construct CAD Generation with Dimension-Aware Parametric Precision](https://mlsanity.com/paper/2606.29301/) (2026)
- [SOV-CAD: Stepwise Orthographic Views Guided CAD Modeling Sequence Reconstruction](https://mlsanity.com/paper/2607.04119/) (2026)
- [CADForge: Agentic Single-View CAD Reconstruction with Explicit Geometry Reasoning](https://mlsanity.com/paper/2610.04262/) (2026)
- [Flatten The Complex: Joint B-Rep Generation via Compositional k-Cell Particles](https://mlsanity.com/paper/2601.17733/) (2026)
- [Classifying Novel 3D-Printed Objects without Retraining: Towards Post-Production Automation in Additive Manufacturing](https://mlsanity.com/paper/2603.07465/) (2026)
- [Learning Hierarchical and Geometry-Aware Graph Representations for Text-to-CAD](https://mlsanity.com/paper/2604.10075/) (2026)
- [Memory-Augmented Reinforcement Learning Agent for CAD Generation](https://mlsanity.com/paper/2605.19748/) (2026)
- [Decomposer: Learning to Decompile Symbolic Music to Programs](https://mlsanity.com/paper/2607.01849/) (2026)
- [From Pixels to PCells: A Neurosymbolic Approach to Photonic Component Creation](https://mlsanity.com/paper/2608.00084/) (2026)
- [TraceCAD: Trace-Guided Repair for Agentic CAD Generation](https://mlsanity.com/paper/2608.03062/) (2026)
- [RA-CAD: Learning Post-Execution Critique for State-Aware Text-to-CAD Generation](https://mlsanity.com/paper/2608.05714/) (2026)
- [RealCAD: Towards Real-World Image-to-CAD Reconstruction under Domain Shift and Parameter Bias](https://mlsanity.com/paper/2608.30617/) (2026)
- [DiVA: Enabling Interactive Digital Life Simulation via Video Models](https://mlsanity.com/paper/2609.13830/) (2026)
- [Multimodal Cultural Heritage Architectural Style Classification for Residential Buildings in the UAE Based on CLIP Embeddings and SVM](https://mlsanity.com/paper/2609.17181/) (2026)
- [VGGT-CAD: Reconstructing Parametric CAD 3D Model with Geometric Grounding](https://mlsanity.com/paper/2609.21225/) (2026)
- [Vision2CAD: A Visual Agent Harness for Explicit Geometry Referencing and Localization in Parametric CAD Modeling](https://mlsanity.com/paper/2609.22688/) (2026)
- [AgenticCADedit: A Stateful, Tool-Mediated Agentic Approach to Multimodal 3D CAD Editing](https://mlsanity.com/paper/2609.29621/) (2026)

## Abstract (arXiv)

Creating CAD digital twins from the physical world is crucial for manufacturing, design, and simulation. However, current methods typically rely on costly 3D scanning with labor-intensive post-processing. To provide a user-friendly design process, we explore the problem of reverse engineering from unconstrained real-world CAD images that can be easily captured by users of all experiences. However, the scarcity of real-world CAD data poses challenges in directly training such models. To tackle these challenges, we propose CADCrafter, an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data while testing on real-world images. To bridge the significant representation disparity between images and parametric CAD models, we introduce a geometry encoder to accurately capture diverse geometric features. Moreover, the texture-invariant properties of the geometric features can also facilitate the generalization to real-world scenarios. Since compiling CAD parameter sequences into explicit CAD models is a non-differentiable process, the network training inherently lacks explicit geometric supervision. To impose geometric validity constraints, we employ direct preference optimization (DPO) to fine-tune our model with the automatic code checker feedback on CAD sequence quality. Furthermore, we collected a real-world dataset, comprised of multi-view images and corresponding CAD command sequence pairs, to evaluate our method. Experimental results demonstrate that our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.