# CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention

arXiv: https://arxiv.org/abs/2402.17678 · HTML: https://mlsanity.com/paper/2402.17678/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [Scan2Cap: Context-aware Dense Captioning in RGB-D Scans](https://mlsanity.com/paper/2012.02206/) (2020)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [CSGNet: Neural Shape Parser for Constructive Solid Geometry](https://mlsanity.com/paper/1712.08290/) (2017)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [BRepNet: A topological message passing system for solid models](https://mlsanity.com/paper/2104.00706/) (2021)
- [Supervised Fitting of Geometric Primitives to 3D Point Clouds](https://mlsanity.com/paper/1811.08988/) (2018)
- [Hierarchical Neural Coding for Controllable CAD Model Generation](https://mlsanity.com/paper/2307.00149/) (2023)
- [Computer-Aided Design as Language](https://mlsanity.com/paper/2105.02769/) (2021)
- [UCSG-Net -- Unsupervised Discovering of Constructive Solid Geometry Tree](https://mlsanity.com/paper/2006.09102/) (2020)
- [PIE-NET: Parametric Inference of Point Cloud Edges](https://mlsanity.com/paper/2007.04883/) (2020)
- [Vitruvion: A Generative Model of Parametric CAD Sketches](https://mlsanity.com/paper/2109.14124/) (2021)
- [CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly](https://mlsanity.com/paper/2104.05652/) (2021)
- [Point2Cyl: Reverse Engineering 3D Objects from Point Clouds to Extrusion Cylinders](https://mlsanity.com/paper/2112.09329/) (2021)
- [Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling](https://mlsanity.com/paper/2107.02389/) (2021)
- [Inferring CAD Modeling Sequences Using Zone Graphs](https://mlsanity.com/paper/2104.03900/) (2021)
- [SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations](https://mlsanity.com/paper/2303.10613/) (2023)
- [D2CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts](https://mlsanity.com/paper/2301.11497/) (2023)
- [ExtrudeNet: Unsupervised Inverse Sketch-and-Extrude for Shape Parsing](https://mlsanity.com/paper/2209.15632/) (2022)
- [HPNet: Deep Primitive Segmentation Using Hybrid Representations](https://mlsanity.com/paper/2105.10620/) (2021)
- [PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds](https://mlsanity.com/paper/2103.02766/) (2021)
- [Neurosymbolic Models for Computer Graphics](https://mlsanity.com/paper/2304.10320/) (2023)
- [CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds](https://mlsanity.com/paper/2109.00113/) (2021)
- [PvDeConv: Point-Voxel Deconvolution for Autoencoding CAD Construction in 3D](https://mlsanity.com/paper/2101.04493/) (2021)
- [SHARP Challenge 2023: Solving CAD History and pArameters Recovery from Point clouds and 3D scans. Overview, Datasets, Metrics, and Baselines](https://mlsanity.com/paper/2308.15966/) (2023)
- [SepicNet: Sharp Edges Recovery by Parametric Inference of Curves in 3D Shapes](https://mlsanity.com/paper/2304.06531/) (2023)

## Built on by (most cited first, 30 of 64)

- [CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM](https://mlsanity.com/paper/2411.04954/) (2024)
- [CAD-Recode: Reverse Engineering CAD Code from Point Clouds](https://mlsanity.com/paper/2412.14042/) (2024)
- [CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs](https://mlsanity.com/paper/2412.19663/) (2024)
- [Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Models](https://mlsanity.com/paper/2501.19054/) (2025)
- [CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers](https://mlsanity.com/paper/2412.13810/) (2024)
- [Img2CAD: Reverse Engineering 3D CAD Models from Images through VLM-Assisted Conditional Factorization](https://mlsanity.com/paper/2408.01437/) (2024)
- [CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images](https://mlsanity.com/paper/2504.04753/) (2025)
- [FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models](https://mlsanity.com/paper/2411.05823/) (2024)
- [TransCAD: A Hierarchical Transformer for CAD Sequence Inference from Point Clouds](https://mlsanity.com/paper/2407.12702/) (2024)
- [Seek-CAD: A Self-refined Generative Modeling for 3D Parametric CAD Using Local Inference via DeepSeek](https://mlsanity.com/paper/2505.17702/) (2025)
- [Text2CAD: Generating Sequential CAD Models from Beginner-to-Expert Level Text Prompts](https://mlsanity.com/paper/2409.17106/) (2024)
- [CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing](https://mlsanity.com/paper/2502.03997/) (2025)
- [MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds](https://mlsanity.com/paper/2508.14879/) (2025)
- [cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning](https://mlsanity.com/paper/2505.22914/) (2025)
- [ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models](https://mlsanity.com/paper/2512.06328/) (2025)
- [Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection](https://mlsanity.com/paper/2603.04337/) (2026)
- [CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation](https://mlsanity.com/paper/2605.10873/) (2026)
- [CAD-Judge: Toward Efficient Morphological Grading and Verification for Text-to-CAD Generation](https://mlsanity.com/paper/2508.04002/) (2025)
- [PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervision](https://mlsanity.com/paper/2407.13394/) (2024)
- [Drawing2CAD: Sequence-to-Sequence Learning for CAD Generation from Vector Drawings](https://mlsanity.com/paper/2508.18733/) (2025)
- [ArtiCAD: Articulated CAD Assembly Design via Multi-Agent Code Generation](https://mlsanity.com/paper/2604.10992/) (2026)
- [MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creation](https://mlsanity.com/paper/2411.17945/) (2024)
- [RLCAD: Reinforcement Learning Training Gym for Revolution Involved CAD Command Sequence Generation](https://mlsanity.com/paper/2503.18549/) (2025)
- [Clarify Before You Draw: Proactive Agents for Robust Text-to-CAD Generation](https://mlsanity.com/paper/2602.03045/) (2026)
- [Towards High-Fidelity CAD Generation via LLM-Driven Program Generation and Text-Based B-Rep Primitive Grounding](https://mlsanity.com/paper/2603.11831/) (2026)
- [DAVINCI: A Single-Stage Architecture for Constrained CAD Sketch Inference](https://mlsanity.com/paper/2410.22857/) (2024)
- [CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation](https://mlsanity.com/paper/2504.20830/) (2025)
- [CADReasoner: Iterative Program Editing for CAD Reverse Engineering](https://mlsanity.com/paper/2603.29847/) (2026)
- [GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models](https://mlsanity.com/paper/2506.10337/) (2025)
- [MiCADangelo: Fine-Grained Reconstruction of Constrained CAD Models from 3D Scans](https://mlsanity.com/paper/2510.23429/) (2025)

## Abstract (arXiv)

Reverse engineering in the realm of Computer-Aided Design (CAD) has been a longstanding aspiration, though not yet entirely realized. Its primary aim is to uncover the CAD process behind a physical object given its 3D scan. We propose CAD-SIGNet, an end-to-end trainable and auto-regressive architecture to recover the design history of a CAD model represented as a sequence of sketch-and-extrusion from an input point cloud. Our model learns visual-language representations by layer-wise cross-attention between point cloud and CAD language embedding. In particular, a new Sketch instance Guided Attention (SGA) module is proposed in order to reconstruct the fine-grained details of the sketches. Thanks to its auto-regressive nature, CAD-SIGNet not only reconstructs a unique full design history of the corresponding CAD model given an input point cloud but also provides multiple plausible design choices. This allows for an interactive reverse engineering scenario by providing designers with multiple next-step choices along with the design process. Extensive experiments on publicly available CAD datasets showcase the effectiveness of our approach against existing baseline models in two settings, namely, full design history recovery and conditional auto-completion from point clouds.