# Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1608.04236 · HTML: https://mlsanity.com/paper/1608.04236/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2016.

## Built on by (most cited first, 30 of 128)

- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers](https://mlsanity.com/paper/2311.15475/) (2023)
- [Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation](https://mlsanity.com/paper/2306.17115/) (2023)
- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [Holistic 3D Scene Understanding from a Single Image with Implicit Representation](https://mlsanity.com/paper/2103.06422/) (2021)
- [ShaRF: Shape-conditioned Radiance Fields from a Single View](https://mlsanity.com/paper/2102.08860/) (2021)
- [Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale](https://mlsanity.com/paper/2412.09548/) (2024)
- [Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations](https://mlsanity.com/paper/2205.02825/) (2022)
- [Deep Implicit Moving Least-Squares Functions for 3D Reconstruction](https://mlsanity.com/paper/2103.12266/) (2021)
- [3D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models](https://mlsanity.com/paper/2212.00842/) (2022)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Fast Point Cloud Generation with Straight Flows](https://mlsanity.com/paper/2212.01747/) (2022)
- [Instruct 3D-to-3D: Text Instruction Guided 3D-to-3D conversion](https://mlsanity.com/paper/2303.15780/) (2023)
- [Category-Level Metric Scale Object Shape and Pose Estimation](https://mlsanity.com/paper/2109.00326/) (2021)
- [Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequences](https://mlsanity.com/paper/2111.12480/) (2021)
- [Density-preserving Deep Point Cloud Compression](https://mlsanity.com/paper/2204.12684/) (2022)
- [RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models](https://mlsanity.com/paper/2407.06938/) (2024)
- [Neural Surface Maps](https://mlsanity.com/paper/2103.16942/) (2021)
- [TexPainter: Generative Mesh Texturing with Multi-view Consistency](https://mlsanity.com/paper/2406.18539/) (2024)
- [Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splatting](https://mlsanity.com/paper/2403.09981/) (2024)
- [3DAC: Learning Attribute Compression for Point Clouds](https://mlsanity.com/paper/2203.09931/) (2022)
- [Self-Supervised Representation Learning for CAD](https://mlsanity.com/paper/2210.10807/) (2022)
- [ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction](https://mlsanity.com/paper/2212.04096/) (2022)
- [NeuralVDB: High-resolution Sparse Volume Representation using Hierarchical Neural Networks](https://mlsanity.com/paper/2208.04448/) (2022)
- [NeuForm: Adaptive Overfitting for Neural Shape Editing](https://mlsanity.com/paper/2207.08890/) (2022)
- [Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policies](https://mlsanity.com/paper/2406.11740/) (2024)
- [3D Mesh Editing using Masked LRMs](https://mlsanity.com/paper/2412.08641/) (2024)

## Abstract (arXiv)

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.