mlsanityData to 14 Oct 2026

Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Models

arXiv 2503.08737 ↗First posted 11 Mar 2025cs.CV cs.AIcited by 6

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Constructing a compressed latent space through a variational autoencoder (VAE) is the key for efficient 3D diffusion models. This paper introduces COD-VAE that encodes 3D shapes into a COmpact set of 1D latent vectors without sacrificing quality. COD-VAE introduces a two-stage autoencoder scheme to improve compression and decoding efficiency. First, our encoder block progressively compresses point clouds into compact latent vectors via intermediate point patches. Second, our triplane-based decoder reconstructs dense triplanes from latent vectors instead of directly decoding neural fields, significantly reducing computational overhead of neural fields decoding. Finally, we propose uncertainty-guided token pruning, which allocates resources adaptively by skipping computations in simpler regions and improves the decoder efficiency. Experimental results demonstrate that COD-VAE achieves 16x compression compared to the baseline while maintaining quality. This enables 20.8x speedup in generation, highlighting that a large number of latent vectors is not a prerequisite for high-quality reconstruction and generation. The code is available at https://github.com/join16/COD-VAE.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 55 of its 64 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

55 of 64 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation21 Jan
  2. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders23 Dec
  3. 2024OctFusion: Octree-based Diffusion Models for 3D Shape Generation27 Aug
  4. 2024DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data6 Jun
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  6. 2024Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking12 Jan
  7. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  8. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  9. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation4 Jul
  10. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  11. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  12. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
5 more
  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  2. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models1 Dec
  3. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  4. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 38

  1. 2016Layer Normalizationcited by 5,669
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  7. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  10. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  11. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  12. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
26 more
  1. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  2. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  3. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
  4. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  5. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 269
  6. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
  7. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  8. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  9. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
  10. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  11. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 130
  12. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  13. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  14. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  15. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 101
  16. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 98
  17. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 93
  18. 2022SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generationcited by 92
  19. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 92
  20. 2020Discrete Point Flow Networks for Efficient Point Cloud Generationcited by 87
  21. 20213D Shape Generation with Grid-based Implicit Functionscited by 48
  22. 2020ManifoldPlus: A Robust and Scalable Watertight Manifold Surface Generation Method for Triangle Soupscited by 38
  23. 2024LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusioncited by 16
  24. 2024GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesiscited by 12
  25. 20243D representation in 512-Byte:Variational tokenizer is the key for autoregressive 3D generationcited by 11
  26. 2025MARS: Mesh AutoRegressive Model for 3D Shape Detailizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026Masked BRep Autoencoder via Hierarchical Graph Transformercited by 3
  2. 2026ZipTok3D: High-Fidelity 3D Tokenization with Compact Token Prefixescited by 1
  3. 2025Squeeze3D: Extreme Neural Compression with Latent Space Bridgingcited by 0
  4. 20253D Shape Generation: A Surveycited by 0
  5. 2026PnP-U3D: Plug-and-Play 3D Framework Bridging Autoregression and Diffusion for Unified Understanding and Generationcited by 0
  6. 2026Generative Shape Reconstruction with Geometry-Guided Langevin Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Geometry-Preserving Encoder/Decoder in Latent Generative Models
  2. 2024Compress3D: a Compressed Latent Space for 3D Generation from a Single Image
  3. 2025Hyper3D: Efficient 3D Representation via Hybrid Triplane and Octree Feature for Enhanced 3D Shape Variational Auto-Encoders
  4. 2023Variational Diffusion Auto-encoder: Latent Space Extraction from Pre-trained Diffusion Models
  5. 2026Class-Partitioned VQ-VAE and Latent Flow Matching for Point Cloud Scene Generation
  6. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space
  7. 2026Geometric Autoencoder for Diffusion Models
  8. 2024Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings