mlsanityData to 14 Oct 2026

CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs

arXiv 2311.16703 ↗First posted 28 Nov 2023cs.CV cs.GRcited by 15

Not on the map. Why?

Abstract (arXiv)

CAD programs are a popular way to compactly encode shapes as a sequence of operations that are easy to parametrically modify. However, without sufficient semantic comments and structure, such programs can be challenging to understand, let alone modify. We introduce the problem of semantic commenting CAD programs, wherein the goal is to segment the input program into code blocks corresponding to semantically meaningful shape parts and assign a semantic label to each block. We solve the problem by combining program parsing with visual-semantic analysis afforded by recent advances in foundational language and vision models. Specifically, by executing the input programs, we create shapes, which we use to generate conditional photorealistic images to make use of semantic annotators for such images. We then distill the information across the images and link back to the original programs to semantically comment on them. Additionally, we collected and annotated a benchmark dataset, CADTalk, consisting of 5,288 machine-made programs and 45 human-made programs with ground truth semantic comments. We extensively evaluated our approach, compared it to a GPT-based baseline, and an open-set shape segmentation baseline, and reported an 83.24% accuracy on the new CADTalk dataset. Code and data: https://enigma-li.github.io/CADTalk/.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

25 of 47 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Segment Anythingcited by 8,234
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2021Grounded Language-Image Pre-trainingcited by 1,156
  9. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  10. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  11. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 304
  12. 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 271
13 more
  1. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 179
  2. 2022Few-shot training LLMs for project-specific code-summarizationcited by 129
  3. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 118
  4. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 87
  5. 2023Hierarchical Neural Coding for Controllable CAD Model Generationcited by 86
  6. 20163D Shape Segmentation with Projective Convolutional Networkscited by 63
  7. 2021CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assemblycited by 62
  8. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  9. 2023Neurosymbolic Models for Computer Graphicscited by 28
  10. 2023ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitivescited by 23
  11. 2023Marching-Primitives: Shape Abstraction from Signed Distance Functioncited by 19
  12. 2021Differentiable 3D CAD Programs for Bidirectional Editingcited by 14
  13. 2022MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 5.

15 papers.

Sort:

By area:

  1. 2025CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generationcited by 67
  2. 2024CAD-Recode: Reverse Engineering CAD Code from Point Cloudscited by 66
  3. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 45
  4. 2024CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solverscited by 40
  5. 2025BlenderGym: Benchmarking Foundational Model Systems for Graphics Editingcited by 31
  6. 2025Large Language Models for Computer-Aided Design: A Surveycited by 18
  7. 2024Text-to-3D Shape Generationcited by 13
  8. 2025CADReview: Automatically Reviewing CAD Programs with Error Detection and Correctioncited by 4
  9. 2026Procedura: Agentic 3D Modeling with Procedural Controlcited by 4
  10. 2025PHT-CAD: Efficient CAD Parametric Primitive Analysis with Progressive Hierarchical Tuningcited by 3
  11. 2025MetaGen: A DSL, Database, and Benchmark for VLM-Assisted Metamaterial Generationcited by 3
  12. 2026BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoningcited by 3
3 more
  1. 2026Symmetry Informative and Agnostic Feature Disentanglement for 3D Shapescited by 2
  2. 2026HierCAD: Hierarchical Text-to-CAD Design via Structure Alignment and Parameter Groundingcited by 1
  3. 2026Task-Adaptive Grounded 3D-Programmers Using 2D VLMscited by 0