mlsanityData to 14 Oct 2026

LangSplat: 3D Language Gaussian Splatting

arXiv 2312.16084 ↗First posted 26 Dec 2023cs.CVcited by 415

Not on the map. Why?

Cited by 21 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Humans live in a 3D world and commonly use natural language to interact with a 3D scene. Modeling a 3D language field to support open-ended language queries in 3D has gained increasing attention recently. This paper introduces LangSplat, which constructs a 3D language field that enables precise and efficient open-vocabulary querying within 3D spaces. Unlike existing methods that ground CLIP language embeddings in a NeRF model, LangSplat advances the field by utilizing a collection of 3D Gaussians, each encoding language features distilled from CLIP, to represent the language field. By employing a tile-based splatting technique for rendering language features, we circumvent the costly rendering process inherent in NeRF. Instead of directly learning CLIP embeddings, LangSplat first trains a scene-wise language autoencoder and then learns language features on the scene-specific latent space, thereby alleviating substantial memory demands imposed by explicit modeling. Existing methods struggle with imprecise and vague 3D language fields, which fail to discern clear boundaries between objects. We delve into this issue and propose to learn hierarchical semantics using SAM, thereby eliminating the need for extensively querying the language field across various scales and the regularization of DINO features. Extensive experimental results show that LangSplat significantly outperforms the previous state-of-the-art method LERF by a large margin. Notably, LangSplat is extremely efficient, achieving a 199 × speedup compared to LERF at the resolution of 1440 × 1080. We strongly recommend readers to check out our video results at https://langsplat.github.io/

Key facts. Cited by 415 papers in the map (data through 14 Oct 2026). Builds on 48 of its 57 references that are in the map. First posted on arXiv 26 Dec 2023.

Builds on

48 of 57 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  8. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  9. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  10. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  11. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  12. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
36 more
  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  2. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 759
  3. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
  4. 2022Language-driven Semantic Segmentationcited by 717
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  7. 2023LERF: Language Embedded Radiance Fieldscited by 543
  8. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 476
  9. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  10. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  11. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  12. 2022Visual Language Maps for Robot Navigationcited by 365
  13. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  14. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  15. 2023Text-to-3D using Gaussian Splattingcited by 299
  16. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  17. 2023Track Anything: Segment Anything Meets Videoscited by 234
  18. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  19. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  20. 2023Segment and Track Anythingcited by 193
  21. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  22. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  23. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 160
  24. 2017IQA: Visual Question Answering in Interactive Environmentscited by 155
  25. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 135
  26. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  27. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 125
  28. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 116
  29. 2022Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesiscited by 97
  30. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 92
  31. 2023On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligencecited by 88
  32. 2023Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusioncited by 64
  33. 2023Segment Anything in 3D with Radiance Fieldscited by 62
  34. 2022OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regressioncited by 44
  35. 2023Matte Anything: Interactive Natural Image Matting with Segment Anything Modelscited by 39
  36. 2023Can SAM Boost Video Super-Resolution?cited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 96; 2025: 173; 2026: 146.

415 papers.

Sort:

By area:

  1. 2024A Survey on 3D Gaussian Splattingcited by 224
  2. 2024OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understandingcited by 145
  3. 2024Recent Advances in 3D Gaussian Splattingcited by 131
  4. 2024Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collectionscited by 117
  5. 2024RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPScited by 113
  6. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  7. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 95
  8. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  9. 2024Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splattingcited by 82
  10. 2024StyleGaussian: Instant 3D Style Transfer with Gaussian Splattingcited by 79
  11. 2024GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Graspingcited by 78
  12. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
38 more
  1. 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
  2. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 71
  3. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 62
  4. 2024MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformationcited by 56
  5. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  6. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
  7. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 54
  8. 2025SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretrainingcited by 52
  9. 2025Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registrationcited by 51
  10. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  11. 2024EgoLifter: Open-world 3D Segmentation for Egocentric Perceptioncited by 47
  12. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  13. 2024REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignmentcited by 45
  14. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 43
  15. 2024SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Realitycited by 43
  16. 2024Gaga: Group Any Gaussians via 3D-aware Memory Bankcited by 41
  17. 2024WaterSplatting: Fast Underwater 3D Scene Reconstruction Using Gaussian Splattingcited by 41
  18. 2024GraspSplats: Efficient Manipulation with 3D Feature Splattingcited by 41
  19. 2025LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPScited by 41
  20. 2024Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splattingcited by 38
  21. 2024LUDVIG: Learning-Free Uplifting of 2D Visual Features to Gaussian Splatting Scenescited by 37
  22. 2024InstanceGaussian: Appearance-Semantic Joint Gaussian Representation for 3D Instance-Level Perceptioncited by 35
  23. 20254D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Modelscited by 35
  24. 2024N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fieldscited by 34
  25. 2024SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussianscited by 33
  26. 2025AutoPartGen: Autogressive 3D Part Generation and Discoverycited by 33
  27. 2024Beyond Bare Queries: Open-Vocabulary Object Grounding with 3D Scene Graphcited by 32
  28. 20243D Vision-Language Gaussian Splattingcited by 31
  29. 2025Rethinking End-to-End 2D to 3D Scene Segmentation in Gaussian Splattingcited by 31
  30. 2024Sort-free Gaussian Splatting via Weighted Sum Renderingcited by 29
  31. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  32. 2024Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstructioncited by 28
  33. 2025IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstructioncited by 28
  34. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 27
  35. 2025Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Imagescited by 26
  36. 2024LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priorscited by 25
  37. 2024GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splattingcited by 25
  38. 2025HRAvatar: High-Quality and Relightable Gaussian Head Avatarcited by 25

50 most cited of 415 All 415 (JSON)