# Learning to Infer 3D Shape Programs with Differentiable Renderer

arXiv: https://arxiv.org/abs/2206.12675 · HTML: https://mlsanity.com/paper/2206.12675/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 25 Jun 2022.

## Builds on (most cited first, 11 of 11)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks](https://mlsanity.com/paper/1708.01648/) (2017)
- [Learning to Infer and Execute 3D Shape Programs](https://mlsanity.com/paper/1901.02875/) (2019)
- [Neural Shape Parsers for Constructive Solid Geometry](https://mlsanity.com/paper/1912.11393/) (2019)
- [Program-Guided Image Manipulators](https://mlsanity.com/paper/1909.02116/) (2019)
- [Multi-Plane Program Induction with 3D Box Priors](https://mlsanity.com/paper/2011.10007/) (2020)
- [Perspective Plane Program Induction from a Single Image](https://mlsanity.com/paper/2006.14708/) (2020)

## Built on by (most cited first, 2 of 2)

- [Real2Code: Reconstruct Articulated Objects via Code Generation](https://mlsanity.com/paper/2406.08474/) (2024)
- [MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds](https://mlsanity.com/paper/2508.14879/) (2025)

## Abstract (arXiv)

Given everyday artifacts, such as tables and chairs, humans recognize high-level regularities within them, such as the symmetries of a table, the repetition of its legs, while possessing low-level priors of their geometries, e.g., surfaces are smooth and edges are sharp. This kind of knowledge constitutes an important part of human perceptual understanding and reasoning. Representations of and how to reason in such knowledge, and the acquisition thereof, are still open questions in artificial intelligence (AI) and cognitive science. Building on the previous proposal of the 3D shape programs representation alone with the accompanying neural generator and executor from citettian2019learning, we propose an analytical yet differentiable executor that is more faithful and controllable in interpreting shape programs (particularly in extrapolation) and more sample efficient (requires no training). These facilitate the generator's learning when ground truth programs are not available, and should be especially useful when new shape-program components are enrolled either by human designers or -- in the context of library learning -- algorithms themselves. Preliminary experiments on using it for adaptation illustrate the aforesaid advantages of the proposed module, encouraging similar methods being explored in building machines that learn to reason with the kind of knowledge described above, and even learn this knowledge itself.