mlsanityData to 7 Oct 2026

GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D Scene Understanding

arXiv 2403.03608 ↗First posted 6 Mar 2024cs.CV cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Utilizing multi-view inputs to synthesize novel-view images, Neural Radiance Fields (NeRF) have emerged as a popular research topic in 3D vision. In this work, we introduce a Generalizable Semantic Neural Radiance Field (GSNeRF), which uniquely takes image semantics into the synthesis process so that both novel view images and the associated semantic maps can be produced for unseen scenes. Our GSNeRF is composed of two stages: Semantic Geo-Reasoning and Depth-Guided Visual rendering. The former is able to observe multi-view image inputs to extract semantic and geometry features from a scene. Guided by the resulting image geometry information, the latter performs both image and semantic rendering with improved performances. Our experiments not only confirm that GSNeRF performs favorably against prior works on both novel-view image and semantic segmentation synthesis but the effectiveness of our sampling strategy for visual rendering is further verified.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 41 of its 46 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

41 of 46 references.

References · 41

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  6. 2015Fast R-CNNcited by 2,930
  7. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  8. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  11. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  12. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
29 more
  1. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  2. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  3. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  4. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  5. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  6. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  7. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  8. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  9. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  10. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  11. 2021GeoNeRF: Generalizing NeRF with Geometry Priorscited by 195
  12. 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
  13. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  14. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  15. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  16. 2021Efficient Neural Radiance Fields for Interactive Free-viewpoint Videocited by 150
  17. 2022Generalizable Patch-Based Neural Renderingcited by 121
  18. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  19. 2022Is Attention All That NeRF Needs?cited by 113
  20. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  21. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  22. 2020Portrait Neural Radiance Fields from a Single Imagecited by 61
  23. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  24. 2023High-Fidelity Clothed Avatar Reconstruction from a Single Imagecited by 38
  25. 2019MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetrycited by 30
  26. 2022RC-MVSNet: Unsupervised Multi-View Stereo with Neural Renderingcited by 30
  27. 2020M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Networkcited by 29
  28. 2023Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attentioncited by 29
  29. 2023ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Imagecited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 4.

15 papers.

Sort:

By area:

  1. 2024RT-GS2: Real-Time Generalizable Semantic Segmentation for 3D Gaussian Representations of Radiance Fieldscited by 5
  2. 2025Understanding while Exploring: Semantics-driven Active Mappingcited by 5
  3. 2026OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understandingcited by 5
  4. 2024SparseLGS: Sparse View Language Embedded Gaussian Splattingcited by 4
  5. 2025Semantic Consistent Language Gaussian Splatting for Point-Level Open-vocabulary Queryingcited by 4
  6. 2025Hi-LSplat: Hierarchical 3D Language Gaussian Splattingcited by 4
  7. 2024SMART: Scene-motion-aware human action recognition framework for mental disorder groupcited by 3
  8. 2025OpenTrack3D: Towards Accurate and Generalizable Open-Vocabulary 3D Instance Segmentationcited by 3
  9. 2025Generalizable Radio-Frequency Radiance Fields for Spatial Spectrum Synthesiscited by 2
  10. 2025UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Renderingcited by 2
  11. 2026SemGS: Feed-Forward Semantic 3D Gaussian Splatting from Sparse Views for Generalizable Scene Understandingcited by 2
  12. 2025GAS-NeRF: Geometry-Aware Stylization of Dynamic Radiance Fieldscited by 0
3 more
  1. 2025UniFField: A Generalizable Unified Neural Feature Field for Visual, Semantic, and Spatial Uncertainties in Any Scenecited by 0
  2. 2026Consistent Scene Understanding in 3D Gaussian Splatting via Multi-Cue Mask Refinementcited by 0
  3. 2026Learning Spectral and Polarimetric Clues for One-to-Multimodal Novel View Synthesiscited by 0