mlsanityData to 7 Oct 2026

Structured 3D Latents for Scalable and Versatile 3D Generation

arXiv 2412.01506 ↗First posted 2 Dec 2024cs.CVcited by 742

Not on the map. Why?

Cited by 137 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a novel 3D generation method for versatile and high-quality 3D asset creation. The cornerstone is a unified Structured LATent (SLAT) representation which allows decoding to different output formats, such as Radiance Fields, 3D Gaussians, and meshes. This is achieved by integrating a sparsely-populated 3D grid with dense multiview visual features extracted from a powerful vision foundation model, comprehensively capturing both structural (geometry) and textural (appearance) information while maintaining flexibility during decoding. We employ rectified flow transformers tailored for SLAT as our 3D generation models and train models with up to 2 billion parameters on a large 3D asset dataset of 500K diverse objects. Our model generates high-quality results with text or image conditions, significantly surpassing existing methods, including recent ones at similar scales. We showcase flexible output format selection and local 3D editing capabilities which were not offered by previous models. Code, model, and data will be released.

Key facts. Cited by 742 papers in the map (data through 7 Oct 2026). Builds on 102 of its 116 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

102 of 116 references.

References · 102

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2023Segment Anythingcited by 8,158
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  12. 2022Scalable Diffusion Models with Transformerscited by 5,931
38 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  11. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  12. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  13. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  14. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  15. 2019Root Mean Square Layer Normalizationcited by 1,425
  16. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  17. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  18. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  19. 2018Demystifying MMD GANscited by 1,268
  20. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  21. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  22. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  23. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  24. 2020Neural Sparse Voxel Fieldscited by 959
  25. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  26. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  27. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  28. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  29. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  30. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  31. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  32. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  33. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 684
  34. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  35. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 661
  36. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  37. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  38. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552

50 most cited of 102 All 102 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 319; 2026: 414.

742 papers.

Sort:

By area:

  1. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  2. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  3. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  4. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 157
  5. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  6. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 120
  7. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 103
  8. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  9. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
  10. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 95
  11. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  12. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 91
38 more
  1. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  2. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  3. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 85
  4. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 84
  5. 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 78
  6. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
  7. 2025SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modelingcited by 76
  8. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 63
  9. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  10. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 60
  11. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 60
  12. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  13. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  14. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  15. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
  16. 2025Any6D: Model-free 6D Pose Estimation of Novel Objectscited by 47
  17. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 47
  18. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  19. 2025Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attentioncited by 43
  20. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 42
  21. 2025LATTICE: Democratize High-Fidelity 3D Generation at Scalecited by 41
  22. 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
  23. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 40
  24. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 40
  25. 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
  26. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
  27. 2025Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesiscited by 33
  28. 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
  29. 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 32
  30. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 30
  31. 2025SynCity: Training-Free Generation of 3D Worldscited by 30
  32. 2025AToken: A Unified Tokenizer for Visioncited by 30
  33. 2025ReconViaGen: Towards Accurate Multi-view 3D Object Reconstruction via Generationcited by 30
  34. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 29
  35. 2025PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policiescited by 29
  36. 2025OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generationcited by 27
  37. 2025Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoningcited by 27
  38. 2025From One to More: Contextual Part Latents for 3D Generationcited by 26

50 most cited of 742 All 742 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Direct and Explicit 3D Generation from a Single Image
  2. 2026Fusion-Aware Direct 3D Gaussian Generation with Structured Patch Latent Flows
  3. 2025ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents
  4. 2025Recent Advances in 3D Object and Scene Generation: A Survey
  5. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models
  6. 2026From Visual Synthesis to Interactive Worlds: Toward Production-Ready 3D Asset Generation
  7. 2026Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale
  8. 2026Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generation