# CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs arXiv: https://arxiv.org/abs/2311.16703 ยท HTML: https://mlsanity.com/paper/2311.16703/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 25 of its 47 references that are in the map. - First posted on arXiv 28 Nov 2023. ## Builds on (most cited first, 25 of 25) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021) - [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022) - [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018) - [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023) - [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021) - [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018) - [Few-shot training LLMs for project-specific code-summarization](https://mlsanity.com/paper/2207.04237/) (2022) - [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022) - [TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models](https://mlsanity.com/paper/2310.13772/) (2023) - [Hierarchical Neural Coding for Controllable CAD Model Generation](https://mlsanity.com/paper/2307.00149/) (2023) - [3D Shape Segmentation with Projective Convolutional Networks](https://mlsanity.com/paper/1612.02808/) (2016) - [CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly](https://mlsanity.com/paper/2104.05652/) (2021) - [SATR: Zero-Shot Semantic Segmentation of 3D Shapes](https://mlsanity.com/paper/2304.04909/) (2023) - [Neurosymbolic Models for Computer Graphics](https://mlsanity.com/paper/2304.10320/) (2023) - [ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives](https://mlsanity.com/paper/2305.05661/) (2023) - [Marching-Primitives: Shape Abstraction from Signed Distance Function](https://mlsanity.com/paper/2303.13190/) (2023) - [Differentiable 3D CAD Programs for Bidirectional Editing](https://mlsanity.com/paper/2110.01182/) (2021) - [MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentation](https://mlsanity.com/paper/2208.08580/) (2022) ## Built on by (most cited first, 15 of 15) - [CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation](https://mlsanity.com/paper/2505.04481/) (2025) - [CAD-Recode: Reverse Engineering CAD Code from Point Clouds](https://mlsanity.com/paper/2412.14042/) (2024) - [Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases](https://mlsanity.com/paper/2403.09675/) (2024) - [CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers](https://mlsanity.com/paper/2412.13810/) (2024) - [BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing](https://mlsanity.com/paper/2504.01786/) (2025) - [Large Language Models for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2505.08137/) (2025) - [Text-to-3D Shape Generation](https://mlsanity.com/paper/2403.13289/) (2024) - [CADReview: Automatically Reviewing CAD Programs with Error Detection and Correction](https://mlsanity.com/paper/2505.22304/) (2025) - [Procedura: Agentic 3D Modeling with Procedural Control](https://mlsanity.com/paper/2608.26238/) (2026) - [PHT-CAD: Efficient CAD Parametric Primitive Analysis with Progressive Hierarchical Tuning](https://mlsanity.com/paper/2503.18147/) (2025) - [MetaGen: A DSL, Database, and Benchmark for VLM-Assisted Metamaterial Generation](https://mlsanity.com/paper/2508.17568/) (2025) - [BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning](https://mlsanity.com/paper/2602.22284/) (2026) - [Symmetry Informative and Agnostic Feature Disentanglement for 3D Shapes](https://mlsanity.com/paper/2601.14804/) (2026) - [HierCAD: Hierarchical Text-to-CAD Design via Structure Alignment and Parameter Grounding](https://mlsanity.com/paper/2607.11339/) (2026) - [Task-Adaptive Grounded 3D-Programmers Using 2D VLMs](https://mlsanity.com/paper/2610.02021/) (2026) ## Abstract (arXiv) CAD programs are a popular way to compactly encode shapes as a sequence of operations that are easy to parametrically modify. However, without sufficient semantic comments and structure, such programs can be challenging to understand, let alone modify. We introduce the problem of semantic commenting CAD programs, wherein the goal is to segment the input program into code blocks corresponding to semantically meaningful shape parts and assign a semantic label to each block. We solve the problem by combining program parsing with visual-semantic analysis afforded by recent advances in foundational language and vision models. Specifically, by executing the input programs, we create shapes, which we use to generate conditional photorealistic images to make use of semantic annotators for such images. We then distill the information across the images and link back to the original programs to semantically comment on them. Additionally, we collected and annotated a benchmark dataset, CADTalk, consisting of 5,288 machine-made programs and 45 human-made programs with ground truth semantic comments. We extensively evaluated our approach, compared it to a GPT-based baseline, and an open-set shape segmentation baseline, and reported an 83.24% accuracy on the new CADTalk dataset. Code and data: https://enigma-li.github.io/CADTalk/.