mlsanityData to 7 Oct 2026

Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review

arXiv 2402.11141 ↗First posted 17 Feb 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers. It explores how NeRFs adeptly infer 3D representations for both stationary and dynamic objects in a scene. This capability is pivotal for generating high-quality new viewpoints, completing missing scene details (inpainting), conducting comprehensive scene segmentation (panoptic segmentation), predicting 3D bounding boxes, editing 3D scenes, and extracting object-centric 3D models. A significant aspect of this study is the application of semantic labels as viewpoint-invariant functions, which effectively map spatial coordinates to a spectrum of semantic labels, thus facilitating the recognition of distinct objects within the scene. Overall, this survey highlights the progression and diverse applications of semantically-aware neural radiance fields in the context of visual scene interpretation.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 251 of its 298 references that are in the map. First posted on arXiv 17 Feb 2024.

Builds on

251 of 298 references.

References · 251

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Segment Anythingcited by 8,158
  6. 2017Mask R-CNNcited by 7,259
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  11. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
38 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  8. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  9. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  10. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  13. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  14. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  15. 2018Demystifying MMD GANscited by 1,268
  16. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  17. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  18. 2021Grounded Language-Image Pre-trainingcited by 1,147
  19. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  20. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  21. 2020Neural Sparse Voxel Fieldscited by 959
  22. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  23. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  24. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  25. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  26. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  27. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  28. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  29. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  30. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  31. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  32. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  33. 2022Language-driven Semantic Segmentationcited by 711
  34. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  35. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  36. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  37. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  38. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672

50 most cited of 251 All 251 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 1.

9 papers.

  1. 2025Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentationcited by 23
  2. 2024NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fieldscited by 20
  3. 2024Leveraging Neural Radiance Fields for Pose Estimation of an Unknown Space Object during Proximity Operationscited by 7
  4. 2025Understanding while Exploring: Semantics-driven Active Mappingcited by 5
  5. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 4
  6. 2025RAZER: Robust Accelerated Zero-Shot 3D Open-Vocabulary Panoptic Reconstruction with Spatio-Temporal Aggregationcited by 2
  7. 2025EscherNet++: Simultaneous Amodal Completion and Scalable View Synthesis through Masked Fine-Tuning and Enhanced Feed-Forward 3D Reconstructioncited by 1
  8. 2025OC-SOP: Enhancing Vision-Based 3D Semantic Occupancy Prediction by Object-Centric Awarenesscited by 0
  9. 2026ActiveLang: Active Open-Vocabulary 3D Mapping with Semantic-Uncertainty-Guided Explorationcited by 0