# Learning to Generate 3D Shapes with Generative Cellular Automata

arXiv: https://arxiv.org/abs/2103.04130 · HTML: https://mlsanity.com/paper/2103.04130/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Learning Gradient Fields for Shape Generation](https://mlsanity.com/paper/2008.06520/) (2020)
- [Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis](https://mlsanity.com/paper/1612.00101/) (2016)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention](https://mlsanity.com/paper/1810.05591/) (2018)
- [How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?](https://mlsanity.com/paper/1511.05101/) (2015)
- [Improved Adversarial Systems for 3D Object Generation and Reconstruction](https://mlsanity.com/paper/1707.09557/) (2017)
- [Multimodal Shape Completion via Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2003.07717/) (2020)
- [Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net](https://mlsanity.com/paper/1711.02282/) (2017)
- [Learning to Generate Samples from Noise through Infusion Training](https://mlsanity.com/paper/1703.06975/) (2017)

## Built on by (most cited first, 22 of 22)

- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies](https://mlsanity.com/paper/2312.03806/) (2023)
- [Growing 3D Artefacts and Functional Machines with Neural Cellular Automata](https://mlsanity.com/paper/2103.08737/) (2021)
- [Attention-based Neural Cellular Automata](https://mlsanity.com/paper/2211.01233/) (2022)
- [GECCO: Geometrically-Conditioned Point Diffusion Models](https://mlsanity.com/paper/2303.05916/) (2023)
- [Probabilistic Implicit Scene Completion](https://mlsanity.com/paper/2204.01264/) (2022)
- [Multi-scale Latent Point Consistency Models for 3D Shape Generation](https://mlsanity.com/paper/2412.19413/) (2024)
- [XDGAN: Multi-Modal 3D Shape Generation in 2D Space](https://mlsanity.com/paper/2210.03007/) (2022)
- [Multimodal Shape Completion via IMLE](https://mlsanity.com/paper/2106.16237/) (2021)
- [Budget-Aware Sequential Brick Assembly with Efficient Constraint Satisfaction](https://mlsanity.com/paper/2210.01021/) (2022)
- [Point Cloud Part Editing: Segmentation, Generation, Assembly, and Selection](https://mlsanity.com/paper/2312.11867/) (2023)
- [Empowered Neural Cellular Automata](https://mlsanity.com/paper/2205.06771/) (2022)
- [Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata](https://mlsanity.com/paper/2406.08292/) (2024)
- [INRFlow: Flow Matching for INRs in Ambient Space](https://mlsanity.com/paper/2412.03791/) (2024)
- [A Continuous-Time Consistency Model for 3D Point Cloud Generation](https://mlsanity.com/paper/2509.01492/) (2025)
- [Probabilistic Shape Completion by Estimating Canonical Factors with Hierarchical VAE](https://mlsanity.com/paper/2212.03370/) (2022)
- [3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability](https://mlsanity.com/paper/2307.14051/) (2023)
- [Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach](https://mlsanity.com/paper/2307.12049/) (2023)
- [ExCellGen: Fast, Controllable, Photorealistic 3D Scene Generation from a Single Real-World Exemplar](https://mlsanity.com/paper/2412.16253/) (2024)
- [Mixtures of Neural Cellular Automata: A Stochastic Framework for Growth Modelling and Self-Organization](https://mlsanity.com/paper/2506.20486/) (2025)
- [DiffeoMorph: Learning to Morph 3D Shapes Using Differentiable Agent-Based Simulations](https://mlsanity.com/paper/2512.17129/) (2025)
- [ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning](https://mlsanity.com/paper/2610.00487/) (2026)

## Abstract (arXiv)

We present a probabilistic 3D generative model, named Generative Cellular Automata, which is able to produce diverse and high quality shapes. We formulate the shape generation process as sampling from the transition kernel of a Markov chain, where the sampling chain eventually evolves to the full shape of the learned distribution. The transition kernel employs the local update rules of cellular automata, effectively reducing the search space in a high-resolution 3D grid space by exploiting the connectivity and sparsity of 3D shapes. Our progressive generation only focuses on the sparse set of occupied voxels and their neighborhood, thus enabling the utilization of an expressive sparse convolutional network. We propose an effective training scheme to obtain the local homogeneous rule of generative cellular automata with sequences that are slightly different from the sampling chain but converge to the full shapes in the training data. Extensive experiments on probabilistic shape completion and shape generation demonstrate that our method achieves competitive performance against recent methods.