# Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis

arXiv: https://arxiv.org/abs/2012.13522 · HTML: https://mlsanity.com/paper/2012.13522/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Dec 2020.

## Built on by (most cited first, 26 of 26)

- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [3DILG: Irregular Latent Grids for 3D Generative Modeling](https://mlsanity.com/paper/2205.13914/) (2022)
- [SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation](https://mlsanity.com/paper/2206.12055/) (2022)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training](https://mlsanity.com/paper/2302.14007/) (2023)
- [A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model](https://mlsanity.com/paper/2205.06924/) (2022)
- [S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic](https://mlsanity.com/paper/2405.00987/) (2024)
- [Learning Energy-Based Prior Model with Diffusion-Amortized MCMC](https://mlsanity.com/paper/2310.03218/) (2023)
- [iFlame: Interleaving Full and Linear Attention for Efficient Mesh Generation](https://mlsanity.com/paper/2503.16653/) (2025)
- [Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation](https://mlsanity.com/paper/2103.04285/) (2021)
- [Energy-Based Generative Cooperative Saliency Prediction](https://mlsanity.com/paper/2106.13389/) (2021)
- [Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability](https://mlsanity.com/paper/2402.12225/) (2024)
- [FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generation](https://mlsanity.com/paper/2603.01515/) (2026)
- [EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models](https://mlsanity.com/paper/2105.04771/) (2021)
- [An Energy-Based Prior for Generative Saliency](https://mlsanity.com/paper/2204.08803/) (2022)
- [EBMs Trained with Maximum Likelihood are Generator Models Trained with a Self-adverserial Loss](https://mlsanity.com/paper/2102.11757/) (2021)
- [CoopInit: Initializing Generative Adversarial Networks via Cooperative Learning](https://mlsanity.com/paper/2303.11649/) (2023)
- [Likelihood-Based Generative Radiance Field with Latent Space Energy-Based Model for 3D-Aware Disentangled Image Representation](https://mlsanity.com/paper/2304.07918/) (2023)
- [Deep Generative Models on 3D Representations: A Survey](https://mlsanity.com/paper/2210.15663/) (2022)
- [Optimizing Implicit Neural Representations from Point Clouds via Energy-Based Models](https://mlsanity.com/paper/2311.02601/) (2023)
- [Incorporating Inductive Biases to Energy-based Generative Models](https://mlsanity.com/paper/2505.01111/) (2025)
- [STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models](https://mlsanity.com/paper/2310.12667/) (2023)
- [A Hybrid Generative and Discriminative PointNet on Unordered Point Sets](https://mlsanity.com/paper/2404.12925/) (2024)
- [Masked Generative Extractor for Synergistic Representation and 3D Generation of Point Clouds](https://mlsanity.com/paper/2406.17342/) (2024)
- [3D-ANC: Adaptive Neural Collapse for Robust 3D Point Cloud Recognition](https://mlsanity.com/paper/2511.07040/) (2025)
- [Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers](https://mlsanity.com/paper/2607.00382/) (2026)

## Abstract (arXiv)

3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world. With the recent emergence of large-scale 3D datasets, it becomes increasingly crucial to have a powerful 3D generative model for 3D shape synthesis and analysis. This paper proposes a deep 3D energy-based model to represent volumetric shapes. The maximum likelihood training of the model follows an "analysis by synthesis" scheme. The benefits of the proposed model are six-fold: first, unlike GANs and VAEs, the model training does not rely on any auxiliary models; second, the model can synthesize realistic 3D shapes by Markov chain Monte Carlo (MCMC); third, the conditional model can be applied to 3D object recovery and super resolution; fourth, the model can serve as a building block in a multi-grid modeling and sampling framework for high resolution 3D shape synthesis; fifth, the model can be used to train a 3D generator via MCMC teaching; sixth, the unsupervisedly trained model provides a powerful feature extractor for 3D data, which is useful for 3D object classification. Experiments demonstrate that the proposed model can generate high-quality 3D shape patterns and can be useful for a wide variety of 3D shape analysis.