mlsanityData to 7 Oct 2026

3D Neural Field Generation using Triplane Diffusion

arXiv 2211.16677 ↗First posted 30 Nov 2022cs.CV cs.AI cs.GRcited by 274

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Diffusion models have emerged as the state-of-the-art for image generation, among other tasks. Here, we present an efficient diffusion-based model for 3D-aware generation of neural fields. Our approach pre-processes training data, such as ShapeNet meshes, by converting them to continuous occupancy fields and factoring them into a set of axis-aligned triplane feature representations. Thus, our 3D training scenes are all represented by 2D feature planes, and we can directly train existing 2D diffusion models on these representations to generate 3D neural fields with high quality and diversity, outperforming alternative approaches to 3D-aware generation. Our approach requires essential modifications to existing triplane factorization pipelines to make the resulting features easy to learn for the diffusion model. We demonstrate state-of-the-art results on 3D generation on several object classes from ShapeNet.

Key facts. Cited by 274 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 76 of its 100 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

76 of 100 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 75

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
38 more
  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  7. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  11. 2020Neural Sparse Voxel Fieldscited by 959
  12. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  13. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  14. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  15. 2016Pixel Recurrent Neural Networkscited by 814
  16. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  17. 2020Convolutional Occupancy Networkscited by 760
  18. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  19. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  20. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  21. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  22. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  23. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  24. 2021Score-based Generative Modeling in Latent Spacecited by 634
  25. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  26. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  27. 2021Neural Fields in Visual Computing and Beyondcited by 539
  28. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  29. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  30. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  31. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  32. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  33. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  34. 2018Assessing Generative Models via Precision and Recallcited by 401
  35. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  36. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  37. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  38. 2021Advances in Neural Renderingcited by 365

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 75; 2024: 116; 2025: 58; 2026: 25.

274 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  4. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 684
  5. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  6. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  8. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  9. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  10. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  11. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  12. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 240
38 more
  1. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  2. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  3. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  4. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  5. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  6. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  7. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  8. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  9. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  10. 2023Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fieldscited by 142
  11. 2023DiffusionSat: A Generative Foundation Model for Satellite Imagerycited by 130
  12. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  13. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 120
  14. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 114
  15. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  16. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  17. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  18. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  19. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  20. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  21. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  22. 2023SKED: Sketch-guided Text-based 3D Editingcited by 72
  23. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  24. 20233D-aware Image Generation using 2D Diffusion Modelscited by 69
  25. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  26. 2024Towards Realistic Scene Generation with LiDAR Diffusion Modelscited by 67
  27. 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulationcited by 66
  28. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  29. 2023NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functionscited by 65
  30. 2024Advances in 3D Generation: A Surveycited by 62
  31. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  32. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  33. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  34. 2024Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstructioncited by 56
  35. 2024Wild-GS: Real-Time Novel View Synthesis from Unconstrained Photo Collectionscited by 56
  36. 2023Spatial Functa: Scaling Functa to ImageNet Classification and Generationcited by 55
  37. 2023Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Modelscited by 53
  38. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53

50 most cited of 274 All 274 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
  2. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models
  3. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generation
  4. 2024Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields
  5. 2023Generating Images with 3D Annotations Using Diffusion Models
  6. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images
  7. 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models
  8. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy