# Autoregressive 3D Shape Generation via Canonical Mapping

arXiv: https://arxiv.org/abs/2204.01955 · HTML: https://mlsanity.com/paper/2204.01955/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 39 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Learning Gradient Fields for Shape Generation](https://mlsanity.com/paper/2008.06520/) (2020)
- [AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation](https://mlsanity.com/paper/2203.09516/) (2022)
- [SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer](https://mlsanity.com/paper/2108.04444/) (2021)
- [Morphing and Sampling Network for Dense Point Cloud Completion](https://mlsanity.com/paper/1912.00280/) (2019)
- [ShapeFormer: Transformer-based Shape Completion via Sparse Representation](https://mlsanity.com/paper/2201.10326/) (2022)
- [PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention](https://mlsanity.com/paper/1810.05591/) (2018)
- [Multiresolution Tree Networks for 3D Point Cloud Processing](https://mlsanity.com/paper/1807.03520/) (2018)

## Built on by (most cited first, 30 of 37)

- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [3DILG: Irregular Latent Grids for 3D Generative Modeling](https://mlsanity.com/paper/2205.13914/) (2022)
- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [Make-A-Shape: a Ten-Million-scale 3D Shape Model](https://mlsanity.com/paper/2401.11067/) (2024)
- [Neural Vector Fields: Implicit Representation by Explicit Learning](https://mlsanity.com/paper/2303.04341/) (2023)
- [Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generation](https://mlsanity.com/paper/2307.03869/) (2023)
- [iFlame: Interleaving Full and Linear Attention for Efficient Mesh Generation](https://mlsanity.com/paper/2503.16653/) (2025)
- [MinD-3D: Reconstruct High-quality 3D objects in Human Brain](https://mlsanity.com/paper/2312.07485/) (2023)
- [Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings](https://mlsanity.com/paper/2411.08017/) (2024)
- [TUVF: Learning Generalizable Texture UV Radiance Fields](https://mlsanity.com/paper/2305.03040/) (2023)
- [A Survey on Computational Solutions for Reconstructing Complete Objects by Reassembling Their Fractured Parts](https://mlsanity.com/paper/2410.14770/) (2024)
- [PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction](https://mlsanity.com/paper/2510.05613/) (2025)
- [Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability](https://mlsanity.com/paper/2402.12225/) (2024)
- [Generating 3D House Wireframes with Semantics](https://mlsanity.com/paper/2407.12267/) (2024)
- [Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Program](https://mlsanity.com/paper/2212.12952/) (2022)
- [MinD-3D++: Advancing fMRI-Based 3D Reconstruction with High-Quality Textured Mesh Generation and a Comprehensive Dataset](https://mlsanity.com/paper/2409.11315/) (2024)
- [Point Cloud Part Editing: Segmentation, Generation, Assembly, and Selection](https://mlsanity.com/paper/2312.11867/) (2023)
- [Learning Versatile 3D Shape Generation with Improved AR Models](https://mlsanity.com/paper/2303.14700/) (2023)
- [FullFormer: Generating Shapes Inside Shapes](https://mlsanity.com/paper/2303.11235/) (2023)
- [Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization](https://mlsanity.com/paper/2309.01512/) (2023)
- [Point Cloud Synthesis Using Inner Product Transforms](https://mlsanity.com/paper/2410.18987/) (2024)
- [3D-WAG: Hierarchical Wavelet-Guided Autoregressive Generation for High-Fidelity 3D Shapes](https://mlsanity.com/paper/2411.19037/) (2024)
- [Rollback-Free Stable Brick Structures Generation](https://mlsanity.com/paper/2605.06947/) (2026)
- [BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenization](https://mlsanity.com/paper/2605.26182/) (2026)
- [Probabilistic Shape Completion by Estimating Canonical Factors with Hierarchical VAE](https://mlsanity.com/paper/2212.03370/) (2022)
- [3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability](https://mlsanity.com/paper/2307.14051/) (2023)
- [Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art](https://mlsanity.com/paper/2311.10129/) (2023)
- [Mitigating Prior Shape Bias in Point Clouds via Differentiable Center Learning](https://mlsanity.com/paper/2402.02088/) (2024)
- [Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenization](https://mlsanity.com/paper/2505.18819/) (2025)

## Abstract (arXiv)

With the capacity of modeling long-range dependencies in sequential data, transformers have shown remarkable performances in a variety of generative tasks such as image, audio, and text generation. Yet, taming them in generating less structured and voluminous data formats such as high-resolution point clouds have seldom been explored due to ambiguous sequentialization processes and infeasible computation burden. In this paper, we aim to further exploit the power of transformers and employ them for the task of 3D point cloud generation. The key idea is to decompose point clouds of one category into semantically aligned sequences of shape compositions, via a learned canonical space. These shape compositions can then be quantized and used to learn a context-rich composition codebook for point cloud generation. Experimental results on point cloud reconstruction and unconditional generation show that our model performs favorably against state-of-the-art approaches. Furthermore, our model can be easily extended to multi-modal shape completion as an application for conditional shape generation.