# GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors

arXiv: https://arxiv.org/abs/2409.16294 · HTML: https://mlsanity.com/paper/2409.16294/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 8 Sep 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024)
- [Shap-E: Generating Conditional 3D Implicit Functions](https://mlsanity.com/paper/2305.02463/) (2023)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [Generating Images with Multimodal Language Models](https://mlsanity.com/paper/2305.17216/) (2023)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation](https://mlsanity.com/paper/1911.02744/) (2019)
- [BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry](https://mlsanity.com/paper/2401.15563/) (2024)
- [MeshNet: Mesh Neural Network for 3D Shape Representation](https://mlsanity.com/paper/1811.11424/) (2018)
- [BRepNet: A topological message passing system for solid models](https://mlsanity.com/paper/2104.00706/) (2021)
- [Computer-Aided Design as Language](https://mlsanity.com/paper/2105.02769/) (2021)
- [SolidGen: An Autoregressive Model for Direct B-rep Synthesis](https://mlsanity.com/paper/2203.13944/) (2022)
- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [SketchGen: Generating Constrained CAD Sketches](https://mlsanity.com/paper/2106.02711/) (2021)
- [ROCA: Robust CAD Model Retrieval and Alignment from a Single Image](https://mlsanity.com/paper/2112.01988/) (2021)
- [PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models](https://mlsanity.com/paper/2312.11417/) (2023)

## Built on by (most cited first, 30 of 38)

- [CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM](https://mlsanity.com/paper/2411.04954/) (2024)
- [SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects](https://mlsanity.com/paper/2410.16499/) (2024)
- [CAD-Coder: An Open-Source Vision-Language Model for Computer-Aided Design Code Generation](https://mlsanity.com/paper/2505.14646/) (2025)
- [CADDreamer: CAD Object Generation from Single-view Images](https://mlsanity.com/paper/2502.20732/) (2025)
- [CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design](https://mlsanity.com/paper/2507.09792/) (2025)
- [CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation](https://mlsanity.com/paper/2512.23333/) (2025)
- [Large Language Models for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2505.08137/) (2025)
- [AutoBrep: Autoregressive B-Rep Generation with Unified Topology and Geometry](https://mlsanity.com/paper/2512.03018/) (2025)
- [From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generation](https://mlsanity.com/paper/2508.10118/) (2025)
- [Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection](https://mlsanity.com/paper/2603.04337/) (2026)
- [CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation](https://mlsanity.com/paper/2605.10873/) (2026)
- [Drawing2CAD: Sequence-to-Sequence Learning for CAD Generation from Vector Drawings](https://mlsanity.com/paper/2508.18733/) (2025)
- [CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation](https://mlsanity.com/paper/2504.20830/) (2025)
- [CADFS: A Big CAD Program Dataset and Framework for Computer-Aided Design with Large Language Models](https://mlsanity.com/paper/2605.01925/) (2026)
- [CADFit: Precise Mesh-to-CAD Program Generation with Hybrid Optimization](https://mlsanity.com/paper/2605.01171/) (2026)
- [PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation](https://mlsanity.com/paper/2607.01883/) (2026)
- [GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback](https://mlsanity.com/paper/2603.27448/) (2026)
- [Physics-in-the-Loop: A Hybrid Agentic Architecture for Validated CAD Engineering Design](https://mlsanity.com/paper/2605.19717/) (2026)
- [UniCAD: A Unified Benchmark and Universal Model for Multi-Modal Multi-Task CAD](https://mlsanity.com/paper/2606.05058/) (2026)
- [GenCAD-Self-Repairing: Feasibility Enhancement for 3D CAD Generation](https://mlsanity.com/paper/2505.23287/) (2025)
- [Autoregressive B-Rep Shape Generation with Parametric Surfaces](https://mlsanity.com/paper/2607.17093/) (2026)
- [View2CAD: Reconstructing View-Centric CAD Models from Single RGB-D Scans](https://mlsanity.com/paper/2504.04000/) (2025)
- [FllumaOne: A Code-Native Multimodal CAD Dataset with Executable Programs and Kernel-Validated Feature Histories](https://mlsanity.com/paper/2606.17696/) (2026)
- [HierCAD: Hierarchical Text-to-CAD Design via Structure Alignment and Parameter Grounding](https://mlsanity.com/paper/2607.11339/) (2026)
- [CADForge: Agentic Single-View CAD Reconstruction with Explicit Geometry Reasoning](https://mlsanity.com/paper/2610.04262/) (2026)
- [BrepGaussian: CAD reconstruction from Multi-View Images with Gaussian Splatting](https://mlsanity.com/paper/2602.21105/) (2026)
- [Learning From Design Procedure To Generate CAD Programs for Data Augmentation](https://mlsanity.com/paper/2603.06894/) (2026)
- [Design-Specification Tiling for ICL-based CAD Code Generation](https://mlsanity.com/paper/2603.12712/) (2026)
- [Agentic AI in Engineering and Manufacturing: Industry Perspectives on Utility, Adoption, Challenges, and Opportunities](https://mlsanity.com/paper/2604.09633/) (2026)
- [SchGen: PCB Schematic Generation with Semantic-Grounded Code Representations](https://mlsanity.com/paper/2605.30345/) (2026)

## Abstract (arXiv)

The creation of manufacturable and editable 3D shapes through Computer-Aided Design (CAD) remains a highly manual and time-consuming task, hampered by the complex topology of boundary representations of 3D solids and unintuitive design tools. While most work in the 3D shape generation literature focuses on representations like meshes, voxels, or point clouds, practical engineering applications demand the modifiability and manufacturability of CAD models and the ability for multi-modal conditional CAD model generation. This paper introduces GenCAD, a generative model that employs autoregressive transformers with a contrastive learning framework and latent diffusion models to transform image inputs into parametric CAD command sequences, resulting in editable 3D shape representations. Extensive evaluations demonstrate that GenCAD significantly outperforms existing state-of-the-art methods in terms of the unconditional and conditional generations of CAD models. Additionally, the contrastive learning framework of GenCAD facilitates the retrieval of CAD models using image queries from large CAD databases, which is a critical challenge within the CAD community. Our results provide a significant step forward in highlighting the potential of generative models to expedite the entire design-to-production pipeline and seamlessly integrate different design modalities.