mlsanityData to 14 Oct 2026

Exploring 3D-aware Latent Spaces for Efficiently Learning Numerous Scenes

arXiv 2403.11678 ↗First posted 18 Mar 2024cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a method enabling the scaling of NeRFs to learn a large number of semantically-similar scenes. We combine two techniques to improve the required training time and memory cost per scene. First, we learn a 3D-aware latent space in which we train Tri-Plane scene representations, hence reducing the resolution at which scenes are learned. Moreover, we present a way to share common information across scenes, hence allowing for a reduction of model complexity to learn a particular scene. Our method reduces effective per-scene memory costs by 44% and per-scene time costs by 86% when training 1000 scenes. Our project page can be found at https://3da-ae.github.io .

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 33 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

30 of 33 references.

References · 30

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  7. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  8. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  10. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  12. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
18 more
  1. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  2. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  4. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  5. 2023LERF: Language Embedded Radiance Fieldscited by 543
  6. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  7. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  8. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  9. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 242
  10. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  11. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  12. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  13. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  14. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 112
  15. 2022Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulationcited by 57
  16. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  17. 2023DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Modelcited by 43
  18. 2023Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representationscited by 5

Built on by

Not cited yet.