# VL-Fields: Towards Language-Grounded Neural Implicit Spatial Representations

arXiv: https://arxiv.org/abs/2305.12427 · HTML: https://mlsanity.com/paper/2305.12427/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 34 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [Urban Radiance Fields](https://mlsanity.com/paper/2111.14643/) (2021)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations](https://mlsanity.com/paper/2209.03494/) (2022)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [Panoptic Lifting for 3D Scene Understanding with Neural Fields](https://mlsanity.com/paper/2212.09802/) (2022)
- [CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory](https://mlsanity.com/paper/2210.05663/) (2022)
- [NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes](https://mlsanity.com/paper/2111.13260/) (2021)
- [DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Images](https://mlsanity.com/paper/2208.07227/) (2022)
- [Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields](https://mlsanity.com/paper/2209.09050/) (2022)
- [Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding](https://mlsanity.com/paper/2210.03043/) (2022)

## Built on by (most cited first, 23 of 23)

- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision](https://mlsanity.com/paper/2311.02077/) (2023)
- [FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding](https://mlsanity.com/paper/2401.01970/) (2024)
- [3D-Mem: 3D Scene Memory for Embodied Exploration and Reasoning](https://mlsanity.com/paper/2411.17735/) (2024)
- [EgoLifter: Open-world 3D Segmentation for Egocentric Perception](https://mlsanity.com/paper/2403.18118/) (2024)
- [When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models](https://mlsanity.com/paper/2405.10255/) (2024)
- [Beyond Bare Queries: Open-Vocabulary Object Grounding with 3D Scene Graph](https://mlsanity.com/paper/2406.07113/) (2024)
- [ODIN: A Single Model for 2D and 3D Segmentation](https://mlsanity.com/paper/2401.02416/) (2024)
- [AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings](https://mlsanity.com/paper/2406.08960/) (2024)
- [Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion](https://mlsanity.com/paper/2507.06230/) (2025)
- [Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors](https://mlsanity.com/paper/2403.14526/) (2024)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [Fast Flow-based Visuomotor Policies via Conditional Optimal Transport Couplings](https://mlsanity.com/paper/2505.01179/) (2025)
- [GOV-NeSF: Generalizable Open-Vocabulary Neural Semantic Fields](https://mlsanity.com/paper/2404.00931/) (2024)
- [OpenOcc: Open Vocabulary 3D Scene Reconstruction via Occupancy Representation](https://mlsanity.com/paper/2403.11796/) (2024)
- [LightSplat: Fast and Memory-Efficient Open-Vocabulary 3D Scene Understanding in Five Seconds](https://mlsanity.com/paper/2603.24146/) (2026)
- [Zero-shot Object-Centric Instruction Following: Integrating Foundation Models with Traditional Navigation](https://mlsanity.com/paper/2411.07848/) (2024)
- [The Temporal Trap: Entanglement in Pre-Trained Visual Representations for Visuomotor Policy Learning](https://mlsanity.com/paper/2502.03270/) (2025)
- [Attentive Feature Aggregation or: How Policies Learn to Stop Worrying about Robustness and Attend to Task-Relevant Visual Cues](https://mlsanity.com/paper/2511.10762/) (2025)
- [OpenLex3D: A Tiered Evaluation Benchmark for Open-Vocabulary 3D Scene Representations](https://mlsanity.com/paper/2503.19764/) (2025)
- [OpenMaskDINO3D : Reasoning 3D Segmentation via Large Language Model](https://mlsanity.com/paper/2506.04837/) (2025)
- [Next Best View Selections for Semantic and Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2512.22771/) (2025)

## Abstract (arXiv)

We present Visual-Language Fields (VL-Fields), a neural implicit spatial representation that enables open-vocabulary semantic queries. Our model encodes and fuses the geometry of a scene with vision-language trained latent features by distilling information from a language-driven segmentation model. VL-Fields is trained without requiring any prior knowledge of the scene object classes, which makes it a promising representation for the field of robotics. Our model outperformed the similar CLIP-Fields model in the task of semantic segmentation by almost 10%.