mlsanityData to 14 Oct 2026

A Study of the Framework and Real-World Applications of Language Embedding for 3D Scene Understanding

arXiv 2508.05064 ↗First posted 7 Aug 2025cs.GR cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Gaussian Splatting has rapidly emerged as a transformative technique for real-time 3D scene representation, offering a highly efficient and expressive alternative to Neural Radiance Fields (NeRF). Its ability to render complex scenes with high fidelity has enabled progress across domains such as scene reconstruction, robotics, and interactive content creation. More recently, the integration of Large Language Models (LLMs) and language embeddings into Gaussian Splatting pipelines has opened new possibilities for text-conditioned generation, editing, and semantic scene understanding. Despite these advances, a comprehensive overview of this emerging intersection has been lacking. This survey presents a structured review of current research efforts that combine language guidance with 3D Gaussian Splatting, detailing theoretical foundations, integration strategies, and real-world use cases. We highlight key limitations such as computational bottlenecks, generalizability, and the scarcity of semantically annotated 3D Gaussian data and outline open challenges and future directions for advancing language-guided 3D scene understanding using Gaussian Splatting.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 128 of its 176 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

128 of 176 references.

References · 128

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2023Segment Anythingcited by 8,234
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2020Scaling Laws for Neural Language Modelscited by 6,586
  12. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
38 more
  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  13. 2018Deep contextualized word representationscited by 2,290
  14. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  15. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  16. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  17. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  18. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  19. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  20. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  21. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  22. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  23. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  24. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  25. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  26. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  27. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  28. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  29. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  30. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
  31. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  32. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  33. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  34. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  35. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 364
  36. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  37. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  38. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262

50 most cited of 128 All 128 (JSON)

Built on by

Not cited yet.