# cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.22914 · HTML: https://mlsanity.com/paper/2505.22914/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 65 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 30 of 50)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [CSGNet: Neural Shape Parser for Constructive Solid Geometry](https://mlsanity.com/paper/1712.08290/) (2017)
- [BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry](https://mlsanity.com/paper/2401.15563/) (2024)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models](https://mlsanity.com/paper/2503.22342/) (2025)
- [BRepNet: A topological message passing system for solid models](https://mlsanity.com/paper/2104.00706/) (2021)
- [Write, Execute, Assess: Program Synthesis with a REPL](https://mlsanity.com/paper/1906.04604/) (2019)
- [Hierarchical Neural Coding for Controllable CAD Model Generation](https://mlsanity.com/paper/2307.00149/) (2023)
- [CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM](https://mlsanity.com/paper/2411.04954/) (2024)
- [SolidGen: An Autoregressive Model for Direct B-rep Synthesis](https://mlsanity.com/paper/2203.13944/) (2022)
- [UCSG-Net -- Unsupervised Discovering of Constructive Solid Geometry Tree](https://mlsanity.com/paper/2006.09102/) (2020)
- [Learning to Infer and Execute 3D Shape Programs](https://mlsanity.com/paper/1901.02875/) (2019)
- [PIE-NET: Parametric Inference of Point Cloud Edges](https://mlsanity.com/paper/2007.04883/) (2020)
- [CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation](https://mlsanity.com/paper/2505.04481/) (2025)
- [CAD-Recode: Reverse Engineering CAD Code from Point Clouds](https://mlsanity.com/paper/2412.14042/) (2024)
- [CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention](https://mlsanity.com/paper/2402.17678/) (2024)
- [CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly](https://mlsanity.com/paper/2104.05652/) (2021)
- [CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward](https://mlsanity.com/paper/2505.19713/) (2025)
- [CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs](https://mlsanity.com/paper/2412.19663/) (2024)
- [CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing](https://mlsanity.com/paper/2108.11305/) (2021)
- [Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Models](https://mlsanity.com/paper/2501.19054/) (2025)

## Built on by (most cited first, 16 of 16)

- [CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation](https://mlsanity.com/paper/2605.10873/) (2026)
- [P3D-Bench: Benchmarking MLLMs for Parametric 3D Generation and Structural Reasoning](https://mlsanity.com/paper/2606.11152/) (2026)
- [CADFS: A Big CAD Program Dataset and Framework for Computer-Aided Design with Large Language Models](https://mlsanity.com/paper/2605.01925/) (2026)
- [Procedura: Agentic 3D Modeling with Procedural Control](https://mlsanity.com/paper/2608.26238/) (2026)
- [Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback](https://mlsanity.com/paper/2605.17448/) (2026)
- [BrepLLM: Enabling Large Language Models to Understand Boundary Representations](https://mlsanity.com/paper/2512.16413/) (2025)
- [Draw it like Euclid: Teaching transformer models to generate CAD profiles using ruler and compass construction steps](https://mlsanity.com/paper/2601.09428/) (2026)
- [BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding](https://mlsanity.com/paper/2606.05515/) (2026)
- [CADENA: Stepwise CAD Reverse Engineering](https://mlsanity.com/paper/2608.00799/) (2026)
- [An Exam for Active Observers](https://mlsanity.com/paper/2607.16165/) (2026)
- [Test-Time Scaling for CAD Generation via Verifier-Free Consensus Selection](https://mlsanity.com/paper/2608.09706/) (2026)
- [MIRAGE-CAD: Construction-Mediated Multimodal Generation of Executable CAD Programs](https://mlsanity.com/paper/2608.28669/) (2026)
- [ConCAD: Constraint-Aware Image-to-CAD Generation with Dual-Granularity Rewards](https://mlsanity.com/paper/2609.34494/) (2026)
- [FACT: Fidelity-Aware Construction of Articulated Twins](https://mlsanity.com/paper/2609.37067/) (2026)
- [CADFather: Autonomous CAD Reconstruction through Coordinated Tool Use](https://mlsanity.com/paper/2610.09127/) (2026)
- [Look Where You Can: Active View Selection for CAD Reconstruction under Occlusion](https://mlsanity.com/paper/2610.11954/) (2026)

## Abstract (arXiv)

Computer-Aided Design (CAD) plays a central role in engineering and manufacturing, making it possible to create precise and editable 3D models. Using a variety of sensor or user-provided data as inputs for CAD reconstruction can democratize access to design applications. However, existing methods typically focus on a single input modality, such as point clouds, images, or text, which limits their generalizability and robustness. Leveraging recent advances in vision-language models (VLM), we propose a multi-modal CAD reconstruction model that simultaneously processes all three input modalities. Inspired by large language model (LLM) training paradigms, we adopt a two-stage pipeline: supervised fine-tuning (SFT) on large-scale procedurally generated data, followed by reinforcement learning (RL) fine-tuning using online feedback, obtained programatically. Furthermore, we are the first to explore RL fine-tuning of LLMs for CAD tasks demonstrating that online RL algorithms such as Group Relative Preference Optimization (GRPO) outperform offline alternatives. In the DeepCAD benchmark, our SFT model outperforms existing single-modal approaches in all three input modalities simultaneously. More importantly, after RL fine-tuning, cadrille sets new state-of-the-art on three challenging datasets, including a real-world one. Code is avaliable at https://github.com/col14m/cadrille .