mlsanityData to 7 Oct 2026

Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding

arXiv 2210.03043 ↗First posted 6 Oct 2022cs.CV cs.LG cs.ROcited by 37

Not on the map. Why?

Abstract (arXiv)

General scene understanding for robotics requires flexible semantic representation, so that novel objects and structures which may not have been known at training time can be identified, segmented and grouped. We present an algorithm which fuses general learned features from a standard pre-trained network into a highly efficient 3D geometric neural field representation during real-time SLAM. The fused 3D feature maps inherit the coherence of the neural field's geometry representation. This means that tiny amounts of human labelling interacting at runtime enable objects or even parts of objects to be robustly and accurately segmented in an open set manner.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

20 of 24 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017Mask R-CNNcited by 7,259
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  8. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  9. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  10. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  11. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  12. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
8 more
  1. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  2. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  3. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  4. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 134
  5. 2019PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Thingscited by 97
  6. 2020Mind the Pad -- CNNs can Develop Blind Spotscited by 41
  7. 2021ILabel: Interactive Neural Scene Labellingcited by 28
  8. 2015SemanticPaint: A Framework for the Interactive Segmentation of 3D Scenescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 11; 2025: 7; 2026: 1.

37 papers.

Sort:

By area:

  1. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
  2. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 361
  3. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  4. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
  5. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
  6. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  7. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 133
  8. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  9. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 73
  10. 2024How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Surveycited by 72
  11. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 70
  12. 2024EgoLifter: Open-world 3D Segmentation for Egocentric Perceptioncited by 46
25 more
  1. 2023ObjectSDF++: Improved Object-Compositional Neural Implicit Surfacescited by 38
  2. 2023D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangementcited by 35
  3. 2023VL-Fields: Towards Language-Grounded Neural Implicit Spatial Representationscited by 23
  4. 2024Neural Fields in Robotics: A Surveycited by 21
  5. 2023RICO: Regularizing the Unobservable for Indoor Compositional Reconstructioncited by 19
  6. 2024SLGaussian: Fast Language Gaussian Splatting in Sparse Viewscited by 17
  7. 2023JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fieldscited by 13
  8. 2023Semantic Attention Flow Fields for Monocular Dynamic Scene Decompositioncited by 12
  9. 2024Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Modelscited by 12
  10. 2025OpenGS-SLAM: Open-Set Dense Semantic SLAM with 3D Gaussian Splatting for Object-Level Scene Understandingcited by 11
  11. 2025Feed-Forward SceneDINO for Unsupervised Semantic Scene Completioncited by 11
  12. 2023NeRF-VO: Real-Time Sparse Visual Odometry with Neural Radiance Fieldscited by 10
  13. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  14. 2023TextSLAM: Visual SLAM with Semantic Planar Text Featurescited by 8
  15. 2023Neural Implicit Vision-Language Feature Fieldscited by 7
  16. 2025DINO-SLAM: DINO-informed RGB-D SLAM for Neural Implicit and Explicit Representationscited by 6
  17. 2024Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinementcited by 5
  18. 2023Managing the unknown: a survey on Open Set Recognition and tangential areascited by 4
  19. 2024Zero-shot Object-Centric Instruction Following: Integrating Foundation Models with Traditional Navigationcited by 4
  20. 2024LOSS-SLAM: Lightweight Open-Set Semantic Simultaneous Localization and Mappingcited by 2
  21. 2025Regularized interpolation in 4D neural fields enables optimization of 3D printed geometriescited by 1
  22. 2025DIAL-GS: Dynamic Instance Aware Reconstruction for Label-free Street Scenes with 4D Gaussian Splattingcited by 1
  23. 2025Next Best View Selections for Semantic and Dynamic 3D Gaussian Splattingcited by 1
  24. 2026Lightweight 3D Feature Pretraining by Bayesian Inversion of 2D Foundation Modelscited by 1
  25. 2025LiLMaps: Learnable Implicit Language Mapscited by 0