# Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration

arXiv: https://arxiv.org/abs/2502.16652 · HTML: https://mlsanity.com/paper/2502.16652/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 49 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [OpenMask3D: Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2306.13631/) (2023)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding](https://mlsanity.com/paper/2311.18482/) (2023)
- [PLA: Language-Driven Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2211.16312/) (2022)
- [OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding](https://mlsanity.com/paper/2406.02058/) (2024)
- [Weakly Supervised 3D Open-vocabulary Segmentation](https://mlsanity.com/paper/2305.14093/) (2023)
- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis](https://mlsanity.com/paper/2404.06270/) (2024)
- [RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding](https://mlsanity.com/paper/2304.00962/) (2023)
- [Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping](https://mlsanity.com/paper/2309.07970/) (2023)

## Built on by (most cited first, 30 of 51)

- [C3G: Learning Compact 3D Representations with 2K Gaussians](https://mlsanity.com/paper/2512.04021/) (2025)
- [A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation](https://mlsanity.com/paper/2508.09977/) (2025)
- [Visibility-Aware Language Aggregation for Open-Vocabulary Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2509.05515/) (2025)
- [COS3D: Collaborative Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2510.20238/) (2025)
- [Inpaint360GS: Efficient Object-Aware 3D Inpainting via Gaussian Splatting for 360deg Scenes](https://mlsanity.com/paper/2511.06457/) (2025)
- [Splat Feature Solver](https://mlsanity.com/paper/2508.12216/) (2025)
- [GALA: Guided Attention with Language Alignment for Open Vocabulary Gaussian Splatting](https://mlsanity.com/paper/2508.14278/) (2025)
- [ExtrinSplat: Decoupling Geometry and Semantics for Open-Vocabulary Understanding in 3D Gaussian Splatting](https://mlsanity.com/paper/2509.22225/) (2025)
- [ReLaGS: Relational Language Gaussian Splatting](https://mlsanity.com/paper/2603.17605/) (2026)
- [RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Models](https://mlsanity.com/paper/2511.19704/) (2025)
- [Consistent Instance Field for Dynamic Scene Understanding](https://mlsanity.com/paper/2512.14126/) (2025)
- [OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2601.09575/) (2026)
- [Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes](https://mlsanity.com/paper/2602.22667/) (2026)
- [LightSplat: Fast and Memory-Efficient Open-Vocabulary 3D Scene Understanding in Five Seconds](https://mlsanity.com/paper/2603.24146/) (2026)
- [Hi-LSplat: Hierarchical 3D Language Gaussian Splatting](https://mlsanity.com/paper/2506.06822/) (2025)
- [GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis](https://mlsanity.com/paper/2603.14965/) (2026)
- [ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2607.18801/) (2026)
- [Patch-wise Retrieval: A Bag of Practical Techniques for Instance-level Matching](https://mlsanity.com/paper/2512.12610/) (2025)
- [SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection](https://mlsanity.com/paper/2601.03024/) (2026)
- [ProFuse: Efficient Cross-View Context Fusion for Open-Vocabulary 3D Gaussian Splatting](https://mlsanity.com/paper/2601.04754/) (2026)
- [Sparse Code Uplifting for Efficient 3D Language Gaussian Splatting](https://mlsanity.com/paper/2605.13600/) (2026)
- [FPGS: Feed-Forward Semantic-aware Photorealistic Style Transfer of Large-Scale Gaussian Splatting](https://mlsanity.com/paper/2503.09635/) (2025)
- [Lang3D-XL: Language Embedded 3D Gaussians for Large-scale Scenes](https://mlsanity.com/paper/2512.07807/) (2025)
- [Split&Splat: Zero-Shot Panoptic Segmentation via Explicit Instance Modeling and 3D Gaussian Splatting](https://mlsanity.com/paper/2602.03809/) (2026)
- [TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentation](https://mlsanity.com/paper/2602.08540/) (2026)
- [EmbodiedSplat: Online Feed-Forward Semantic 3DGS for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2603.04254/) (2026)
- [LESV: Language Embedded Sparse Voxel Fusion for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2604.01388/) (2026)
- [OpenGaFF: Open-Vocabulary Gaussian Feature Field with Codebook Attention](https://mlsanity.com/paper/2605.06088/) (2026)
- [TrackRef3D: Multi-View Consistent Track-then-Label for Open-World Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2605.26576/) (2026)
- [Relation-Centric Open-Vocabulary 3D Gaussian Segmentation](https://mlsanity.com/paper/2607.01140/) (2026)

## Abstract (arXiv)

We introduce Dr. Splat, a novel approach for open-vocabulary 3D scene understanding leveraging 3D Gaussian Splatting. Unlike existing language-embedded 3DGS methods, which rely on a rendering process, our method directly associates language-aligned CLIP embeddings with 3D Gaussians for holistic 3D scene understanding. The key of our method is a language feature registration technique where CLIP embeddings are assigned to the dominant Gaussians intersected by each pixel-ray. Moreover, we integrate Product Quantization (PQ) trained on general large-scale image data to compactly represent embeddings without per-scene optimization. Experiments demonstrate that our approach significantly outperforms existing approaches in 3D perception benchmarks, such as open-vocabulary 3D semantic segmentation, 3D object localization, and 3D object selection tasks. For video results, please visit : https://drsplat.github.io/