mlsanityData to 14 Oct 2026

ReferSplat: Referring Segmentation in 3D Gaussian Splatting

arXiv 2508.08252 ↗First posted 11 Aug 2025cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

We introduce Referring 3D Gaussian Splatting Segmentation (R3DGS), a new task that aims to segment target objects in a 3D Gaussian scene based on natural language descriptions, which often contain spatial relationships or object attributes. This task requires the model to identify newly described objects that may be occluded or not directly visible in a novel view, posing a significant challenge for 3D multi-modal understanding. Developing this capability is crucial for advancing embodied AI. To support research in this area, we construct the first R3DGS dataset, Ref-LERF. Our analysis reveals that 3D multi-modal understanding and spatial relationship modeling are key challenges for R3DGS. To address these challenges, we propose ReferSplat, a framework that explicitly models 3D Gaussian points with natural language expressions in a spatially aware paradigm. ReferSplat achieves state-of-the-art performance on both the newly proposed R3DGS task and 3D open-vocabulary segmentation benchmarks. Dataset and code are available at https://github.com/heshuting555/ReferSplat.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 52 of its 58 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

52 of 58 references.

References · 52

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  8. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
  9. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  10. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  11. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 759
  12. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
38 more
  1. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
  2. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
  3. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  4. 2023LERF: Language Embedded Radiance Fieldscited by 543
  5. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  6. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  7. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 393
  8. 2023SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMcited by 388
  9. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  10. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 364
  11. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 286
  12. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  13. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  14. 2024Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenescited by 232
  15. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  16. 2023Language Embedded 3D Gaussians for Open-Vocabulary Scene Understandingcited by 208
  17. 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 179
  18. 2023Segment Any 3D Gaussianscited by 161
  19. 2024High-quality Surface Reconstruction using Gaussian Surfelscited by 161
  20. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  21. 2024OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understandingcited by 145
  22. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  23. 2024HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splattingcited by 142
  24. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 111
  25. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  26. 2024Click-Gaussian: Interactive Segmentation to Any 3D Gaussianscited by 58
  27. 2024GOI: Find 3D Gaussians of Interest with an Optimizable Open-vocabulary Semantic-space Hyperplanecited by 53
  28. 2024GeoGaussian: Geometry-aware Gaussian Splatting for Scene Renderingcited by 52
  29. 2024Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splattingcited by 38
  30. 2024SegPoint: Segment Any Point Cloud via Large Language Modelcited by 35
  31. 2024InstanceGaussian: Appearance-Semantic Joint Gaussian Representation for 3D Instance-Level Perceptioncited by 35
  32. 2024FastLGS: Speeding up Language Embedded Gaussians with Feature Grid Mappingcited by 34
  33. 2024SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussianscited by 33
  34. 20243D Vision-Language Gaussian Splattingcited by 31
  35. 2023Prototype Adaption and Projection for Few- and Zero-shot 3D Point Cloud Semantic Segmentationcited by 26
  36. 2024RefMask3D: Language-Guided Transformer for 3D Referring Segmentationcited by 25
  37. 2024GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splattingcited by 25
  38. 2025Multimodal Referring Segmentation: A Surveycited by 22

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 19.

20 papers.

Sort:

By area:

  1. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  2. 2026OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understandingcited by 5
  3. 2026ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splattingcited by 3
  4. 2026EnerGS: Energy-Based Gaussian Splatting with Partial Geometric Priorscited by 2
  5. 2026VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inferencecited by 2
  6. 2026PanopticQuery: Unified Query-Time Reasoning for 4D Scenescited by 1
  7. 20263D Segmentation Using Viewpoint-Dependent Spatial Relationshipscited by 1
  8. 2026TrackRef3D: Multi-View Consistent Track-then-Label for Open-World Referring Segmentation in 3D Gaussian Splattingcited by 1
  9. 2026Open-Vocabulary and Referring Segmentation for 3D Gaussians Using 2D Detectorscited by 1
  10. 2026G2P: Gaussian-to-Point Attribute Alignment for Boundary-Aware 3D Segmentationcited by 0
  11. 2026TrianguLang: Geometry-Aware Semantic Consensus for Pose-Free 3D Localizationcited by 0
  12. 2026SAD-GS: Learning Reliable 3D Semantic Gaussian Fields via Dynamic Geo-Semantic Anchoringcited by 0
8 more
  1. 2026MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshescited by 0
  2. 2026Seek to Segment: Active Perception for Panoramic Referring Segmentationcited by 0
  3. 2026E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embeddingcited by 0
  4. 2026Beyond Similarity Matching: Structured Reasoning for Open-Vocabulary Referring Segmentation in 3DGScited by 0
  5. 2026Dyn-3D: Unveiling and Resolving Ego-Motion Ambiguity in Vision-Language Modelscited by 0
  6. 2026CoRef-GS: Cooperative Referring Gaussian Splatting for Multi-Agent Scene Understandingcited by 0
  7. 2026DSSR-3D: Decoupled Reasoning for View-Dependent Referring in 3D Gaussianscited by 0
  8. 2026RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentationcited by 0