# ShapeLib: Designing a library of programmatic 3D shape abstractions with Large Language Models

arXiv: https://arxiv.org/abs/2502.08884 · HTML: https://mlsanity.com/paper/2502.08884/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 69 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [CSGNet: Neural Shape Parser for Constructive Solid Geometry](https://mlsanity.com/paper/1712.08290/) (2017)
- [CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM](https://mlsanity.com/paper/2411.04954/) (2024)
- [UCSG-Net -- Unsupervised Discovering of Constructive Solid Geometry Tree](https://mlsanity.com/paper/2006.09102/) (2020)
- [CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing](https://mlsanity.com/paper/2108.11305/) (2021)
- [LILO: Learning Interpretable Libraries by Compressing and Documenting Code](https://mlsanity.com/paper/2310.19791/) (2023)
- [DECOR-GAN: 3D Shape Detailization by Conditional Refinement](https://mlsanity.com/paper/2012.09159/) (2020)
- [Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences](https://mlsanity.com/paper/2010.02392/) (2020)
- [The Scene Language: Representing Scenes with Programs, Words, and Embeddings](https://mlsanity.com/paper/2410.16770/) (2024)
- [BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement](https://mlsanity.com/paper/2412.14203/) (2024)
- [FlairGPT: Repurposing LLMs for Interior Designs](https://mlsanity.com/paper/2501.04648/) (2025)
- [babble: Learning Better Abstractions with E-Graphs and Anti-Unification](https://mlsanity.com/paper/2212.04596/) (2022)
- [ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives](https://mlsanity.com/paper/2305.05661/) (2023)
- [SceneMotifCoder: Example-driven Visual Program Learning for Generating 3D Object Arrangements](https://mlsanity.com/paper/2408.02211/) (2024)
- [PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs](https://mlsanity.com/paper/2308.05744/) (2023)
- [Spice-E : Structural Priors in 3D Diffusion using Cross-Entity Attention](https://mlsanity.com/paper/2311.17834/) (2023)
- [Learning to Edit Visual Programs with Self-Supervision](https://mlsanity.com/paper/2406.02383/) (2024)
- [ParSEL: Parameterized Shape Editing with Language](https://mlsanity.com/paper/2405.20319/) (2024)

## Built on by (most cited first, 5 of 5)

- [Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks](https://mlsanity.com/paper/2510.25760/) (2025)
- [ProcFunc: Function-Oriented Abstractions for Procedural 3D Generation in Python](https://mlsanity.com/paper/2604.26943/) (2026)
- [Thinking with Patterns: Breaking the Perceptual Bottleneck in Visual Planning via Pattern Induction](https://mlsanity.com/paper/2605.16848/) (2026)
- [Can Agents Design Libraries for Agents?](https://mlsanity.com/paper/2609.36730/) (2026)
- [FACT: Fidelity-Aware Construction of Articulated Twins](https://mlsanity.com/paper/2609.37067/) (2026)

## Abstract (arXiv)

We present ShapeLib, the first method that uses the priors of Large Language Models (LLMs) to design libraries of programmatic 3D shape abstractions. Our system accepts two forms of user-provided design intent: high-level text descriptions of functions to include in the output library and a small seed set of exemplar shapes. We discover a library of abstractions that matches this design intent with a guided LLM workflow that first proposes different ways of applying and implementing functions, and then validates these functions are helpful in representing seed set shapes. To extend beyond the seed set, we develop library-specific recognition networks that map shapes (represented as primitives, voxels, or point clouds) to programs that use these newly discovered abstractions. Across multiple modeling domains (split by shape category), we find that LLMs, when thoughtfully combined with geometric reasoning, can be guided to author libraries of abstraction functions that generalize across shape distributions. Our framework takes a step towards realizing the long-standing shape analysis aspiration of discovering reusable, programmatic shape abstractions while exposing interpretable, semantically aligned interfaces. Our extensive evaluation demonstrates that ShapeLib provides distinct advantages over prior alternative abstraction discovery works in terms of generalization, usability, and maintaining plausibility under manipulation. Finally, we demonstrate that ShapeLib's abstraction functions unlock a number of downstream applications, combining LLM reasoning over shape programs with geometry processing tools to support shape editing and generation workflows.