mlsanityData to 7 Oct 2026

Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

arXiv 2412.17808 ↗First posted 23 Dec 2024cs.CVcited by 93

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recent 3D content generation pipelines commonly employ Variational Autoencoders (VAEs) to encode shapes into compact latent representations for diffusion-based generation. However, the widely adopted uniform point sampling strategy in Shape VAE training often leads to a significant loss of geometric details, limiting the quality of shape reconstruction and downstream generation tasks. We present Dora-VAE, a novel approach that enhances VAE reconstruction through our proposed sharp edge sampling strategy and a dual cross-attention mechanism. By identifying and prioritizing regions with high geometric complexity during training, our method significantly improves the preservation of fine-grained shape features. Such sampling strategy and the dual attention mechanism enable the VAE to focus on crucial geometric details that are typically missed by uniform sampling approaches. To systematically evaluate VAE reconstruction quality, we additionally propose Dora-bench, a benchmark that quantifies shape complexity through the density of sharp edges, introducing a new metric focused on reconstruction accuracy at these salient geometric features. Extensive experiments on the Dora-bench demonstrate that Dora-VAE achieves comparable reconstruction quality to the state-of-the-art dense XCube-VAE while requiring a latent space at least 8× smaller (1,280 vs. > 10,000 codes).

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 55 of its 65 references that are in the map. First posted on arXiv 23 Dec 2024.

Builds on

55 of 65 references.

Closest ancestors · 19

Same direction, newest first

  1. 2024OctFusion: Octree-based Diffusion Models for 3D Shape Generation27 Aug
  2. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Model19 Aug
  3. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  4. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner23 May
  5. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer23 May
  6. 2023Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior11 Dec
  7. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  8. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  9. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  10. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  11. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  12. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
7 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  2. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models3 Apr
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  4. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
  5. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  7. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 36

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  9. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  10. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  12. 2016Training Deep Nets with Sublinear Memory Costcited by 825
24 more
  1. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  2. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  3. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  4. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  5. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  6. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  7. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  8. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  9. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  10. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  11. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  12. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  13. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  14. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Modelscited by 139
  15. 2023Neural Kernel Surface Reconstructioncited by 102
  16. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  17. 2022TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decompositioncited by 83
  18. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 56
  19. 2023Attention-based Point Cloud Edge Samplingcited by 37
  20. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  21. 2018Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)cited by 23
  22. 2024fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligencecited by 23
  23. 2024LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusioncited by 16
  24. 2023TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshescited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 43; 2026: 50.

93 papers.

Sort:

By area:

  1. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  2. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 157
  3. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  4. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 103
  5. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 95
  6. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 85
  7. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 84
  8. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
  9. 2025SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modelingcited by 76
  10. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  11. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 47
  12. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 42
38 more
  1. 2025LATTICE: Democratize High-Fidelity 3D Generation at Scalecited by 41
  2. 2025AutoPartGen: Autogressive 3D Part Generation and Discoverycited by 33
  3. 2025ReconViaGen: Towards Accurate Multi-view 3D Object Reconstruction via Generationcited by 30
  4. 2025Unleashing Vecset Diffusion Model for Fast Shape Generationcited by 25
  5. 2025UniTEX: Universal High Fidelity Generative Texturing for 3D Shapescited by 23
  6. 2025Particulate: Feed-Forward 3D Object Articulationcited by 20
  7. 2025Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assetscited by 19
  8. 2026ShapeR: Robust Conditional 3D Shape Generation from Casual Capturescited by 19
  9. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  10. 2025UltraShape 1.0: High-Fidelity 3D Shape Generation via Scalable Geometric Refinementcited by 16
  11. 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 15
  12. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusioncited by 15
  13. 2025Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenizationcited by 12
  14. 2026HY3D-Bench: Generation of 3D Assetscited by 11
  15. 2025UniPart: Part-Level 3D Generation with Unified 3D Geom-Seg Latentscited by 9
  16. 2025Light of Normals: Unified Feature Representation for Universal Photometric Stereocited by 8
  17. 2025SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstructioncited by 8
  18. 2025Hyper3D: Efficient 3D Representation via Hybrid Triplane and Octree Feature for Enhanced 3D Shape Variational Auto-Encoderscited by 7
  19. 2025Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Modelscited by 6
  20. 2025Unified Cross-Scale 3D Generation and Understanding via Autoregressive Modelingcited by 5
  21. 2025ViewCraft3D: High-Fidelity and View-Consistent 3D Vector Graphics Synthesiscited by 5
  22. 2025PhysGen: Physically Grounded 3D Shape Generation for Industrial Designcited by 5
  23. 2025Faithful Contouring: Near-Lossless 3D Voxel Representation Free from Iso-surfacecited by 4
  24. 2025TEXTRIX: Latent Attribute Grid for Native Texture Generation and Beyondcited by 4
  25. 2026MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphingcited by 4
  26. 2026Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generationcited by 4
  27. 2025SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generationcited by 2
  28. 2025Few-step Flow for 3D Generation via Marginal-Data Transport Distillationcited by 2
  29. 2025LoG3D: Ultra-High-Resolution 3D Shape Modeling via Local-to-Global Partitioningcited by 2
  30. 2025UniArt: Unified 3D Representation for Generating 3D Articulated Objects with Open-Set Articulationcited by 2
  31. 2026LiTo: Surface Light Field Tokenizationcited by 2
  32. 2026TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unificationcited by 2
  33. 2026Hitem3D 2.0: Multi-View Guided Native 3D Texture Generationcited by 2
  34. 2026DVD: Discrete Voxel Diffusion for 3D Generation and Editingcited by 2
  35. 2026TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fieldscited by 2
  36. 2026TanGO: Training-Free 3D Editing via Tangent-Space Guidance and Optimizationcited by 2
  37. 2025Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Frameworkcited by 1
  38. 2025Memorization in 3D Shape Generation: An Empirical Studycited by 1

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 20233DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generation
  3. 2024Compress3D: a Compressed Latent Space for 3D Generation from a Single Image
  4. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priors
  5. 2025Geometry-Preserving Encoder/Decoder in Latent Generative Models
  6. 2025OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generation
  7. 2025BridgeShape: Latent Diffusion Schr"odinger Bridge for 3D Shape Completion
  8. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models