mlsanityData to 7 Oct 2026

Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequences

arXiv 2111.12480 ↗First posted 24 Nov 2021cs.CV cs.GR cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

Autoregressive models have proven to be very powerful in NLP text generation tasks and lately have gained popularity for image generation as well. However, they have seen limited use for the synthesis of 3D shapes so far. This is mainly due to the lack of a straightforward way to linearize 3D data as well as to scaling problems with the length of the resulting sequences when describing complex shapes. In this work we address both of these problems. We use octrees as a compact hierarchical shape representation that can be sequentialized by traversal ordering. Moreover, we introduce an adaptive compression scheme, that significantly reduces sequence lengths and thus enables their effective generation with a transformer, while still allowing fully autoregressive sampling and parallel training. We demonstrate the performance of our model by comparing against the state-of-the-art in shape generation.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

31 of 45 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  11. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  12. 2016Pixel Recurrent Neural Networkscited by 814
19 more
  1. 2020Efficient Transformers: A Surveycited by 784
  2. 2018Image Transformercited by 770
  3. 2020Convolutional Occupancy Networkscited by 760
  4. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  5. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  6. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  7. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  8. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  9. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  10. 2020Fast Transformers with Clustered Attentioncited by 123
  11. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  12. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  13. 2018Point Cloud GANcited by 79
  14. 2019StructureNet: Hierarchical Graph Networks for 3D Shape Generationcited by 59
  15. 2016VConv-DAE: Deep Volumetric Shape Learning Without Object Labelscited by 50
  16. 20213D Shape Generation with Grid-based Implicit Functionscited by 48
  17. 2018Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapescited by 25
  18. 2013A new graph-based two-sample test for multivariate and object datacited by 24
  19. 2019A Convolutional Decoder for Point Clouds using Adaptive Instance Normalizationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 11; 2024: 9; 2025: 7; 2026: 6.

38 papers.

Sort:

By area:

  1. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  3. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  4. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  5. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  6. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  7. 2022SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generationcited by 92
  8. 2025RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assetscited by 47
  9. 2024Autoregressive Models in Vision: A Surveycited by 39
  10. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 38
  11. 2025OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generationcited by 27
  12. 2025From One to More: Contextual Part Latents for 3D Generationcited by 26
26 more
  1. 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
  2. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  3. 2023Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priorscited by 24
  4. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusioncited by 22
  5. 2023CAD: Photorealistic 3D Generation via Adversarial Distillationcited by 18
  6. 2024PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformerscited by 14
  7. 2025Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenizationcited by 12
  8. 2023Pushing the Limits of 3D Shape Generation at Scalecited by 11
  9. 2023MinD-3D: Reconstruct High-quality 3D objects in Human Braincited by 11
  10. 2023Progressive Text-to-3D Generation for Automatic 3D Prototypingcited by 8
  11. 2024Generating 3D House Wireframes with Semanticscited by 8
  12. 2024MinD-3D++: Advancing fMRI-Based 3D Reconstruction with High-Quality Textured Mesh Generation and a Comprehensive Datasetcited by 6
  13. 2025Unified Cross-Scale 3D Generation and Understanding via Autoregressive Modelingcited by 5
  14. 2024Adaptive Patching for High-resolution Image Segmentation with Transformerscited by 4
  15. 2026GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generationcited by 4
  16. 2025MFM-point: Multi-scale Flow Matching for Point Cloud Generationcited by 3
  17. 2026Repurposing 3D Generative Model for Autoregressive Layout Generationcited by 3
  18. 2024Multidimensional Byte Pair Encoding: Shortened Sequences for Improved Visual Data Generationcited by 2
  19. 2022Deep Generative Models on 3D Representations: A Surveycited by 1
  20. 2024PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Imagescited by 1
  21. 2024GALA: Geometry-Aware Local Adaptive Grids for Detailed 3D Generationcited by 1
  22. 20253D Shape Generation: A Surveycited by 0
  23. 2026MegaParts: Scaling Part-Aware 3D Object Generation to 300 Parts via Token-Efficient Autoregressive Modelingcited by 0
  24. 2026aDSL: Agentic 3D Creation via Joint Agent-Program Designcited by 0
  25. 2026ToW3D: Consistency-aware Interactive Point-based Mesh Editing on GANscited by 0
  26. 2026Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understandingcited by 0