A Study of the Framework and Real-World Applications of Language Embedding for 3D Scene Understanding
Not on the map. Why?
Abstract (arXiv)
Gaussian Splatting has rapidly emerged as a transformative technique for real-time 3D scene representation, offering a highly efficient and expressive alternative to Neural Radiance Fields (NeRF). Its ability to render complex scenes with high fidelity has enabled progress across domains such as scene reconstruction, robotics, and interactive content creation. More recently, the integration of Large Language Models (LLMs) and language embeddings into Gaussian Splatting pipelines has opened new possibilities for text-conditioned generation, editing, and semantic scene understanding. Despite these advances, a comprehensive overview of this emerging intersection has been lacking. This survey presents a structured review of current research efforts that combine language guidance with 3D Gaussian Splatting, detailing theoretical foundations, integration strategies, and real-world use cases. We highlight key limitations such as computational bottlenecks, generalizability, and the scarcity of semantically annotated 3D Gaussian data and outline open challenges and future directions for advancing language-guided 3D scene understanding using Gaussian Splatting.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 128 of its 176 references that are in the map. First posted on arXiv 7 Aug 2025.
Builds on
128 of 176 references.
References · 128
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
38 more
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2018Deep contextualized word representationscited by 2,290
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
- 2016Enriching Word Vectors with Subword Informationcited by 1,583
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
- 2024Grounding Image Matching in 3D with MASt3Rcited by 983
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
- 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
- 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
- 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
- 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 364
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
50 most cited of 128 All 128 (JSON)
Built on by
Not cited yet.