mlsanityData to 14 Oct 2026

A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions

arXiv 2406.05785 ↗First posted 9 Jun 2024cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

Text-guided 3D visual grounding (T-3DVG), which aims to locate a specific object that semantically corresponds to a language query from a complicated 3D scene, has drawn increasing attention in the 3D research community over the past few years. Compared to 2D visual grounding, this task presents great potential and challenges due to its closer proximity to the real world and the complexity of data collection and 3D point cloud source processing. In this survey, we attempt to provide a comprehensive overview of the T-3DVG progress, including its fundamental elements, recent research advances, and future research directions. To the best of our knowledge, this is the first systematic survey on the T-3DVG task. Specifically, we first provide a general structure of the T-3DVG pipeline with detailed components in a tutorial style, presenting a complete background overview. Then, we summarize the existing T-3DVG approaches into different categories and analyze their strengths and weaknesses. We also present the benchmark datasets and evaluation metrics to assess their performances. Finally, we discuss the potential limitations of existing T-3DVG and share some insights on several promising research directions. The latest papers are continually collected at https://github.com/liudaizong/Awesome-3D-Visual-Grounding.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 115 of its 195 references that are in the map. First posted on arXiv 9 Jun 2024.

Builds on

115 of 195 references.

References · 115

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  9. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  10. 2017Graph Attention Networkscited by 8,923
  11. 2023Visual Instruction Tuningcited by 7,869
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
38 more
  1. 2017Inductive Representation Learning on Large Graphscited by 6,320
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  9. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  13. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  14. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  15. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  16. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  17. 2018Panoptic Segmentationcited by 802
  18. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  19. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  20. 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
  21. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  22. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  23. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 328
  24. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 321
  25. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  26. 2019Disentangling Monocular 3D Object Detectioncited by 217
  27. 2021Group-Free 3D Object Detection via Transformerscited by 215
  28. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  29. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  30. 2020Dense Regression Network for Video Groundingcited by 176
  31. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  32. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  33. 2020EPNet: Enhancing Point Features with Image Semantics for 3D Object Detectioncited by 164
  34. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  35. 2021NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstructioncited by 146
  36. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  37. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  38. 2022Multi-View Transformer for 3D Visual Groundingcited by 140

50 most cited of 115 All 115 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 13; 2026: 6.

25 papers.

Sort:

By area:

  1. 2024Harmonizing Visual Text Comprehension and Generationcited by 54
  2. 2024Toward Visual Grounding: A Surveycited by 52
  3. 2025Multimodal Referring Segmentation: A Surveycited by 22
  4. 2025TSP3D: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Groundingcited by 17
  5. 2025SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Groundingcited by 12
  6. 2024Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problemscited by 7
  7. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
  8. 2025S2-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidancecited by 5
  9. 2025UWBench: A Comprehensive Vision-Language Benchmark for Underwater Understandingcited by 4
  10. 2024Joint Top-Down and Bottom-Up Frameworks for 3D Visual Groundingcited by 3
  11. 2025Descrip3D: Enhancing Large Language Model-based 3D Scene Understanding with Object-Level Text Descriptionscited by 3
  12. 2025PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Groundingcited by 3
13 more
  1. 2024Fine-Grained Spatial and Verbal Losses for 3D Visual Groundingcited by 1
  2. 2025Visual Grounding from Event Camerascited by 1
  3. 2026From Part to Whole: 3D Generative World Model with an Adaptive Structural Hierarchycited by 1
  4. 2024Hard-Label Black-Box Attacks on 3D Point Cloudscited by 0
  5. 2025First-place Solution for Streetscape Shop Sign Recognition Competitioncited by 0
  6. 2025Spontaneous emergence of linguistic statistical laws in images via artificial neural networkscited by 0
  7. 2025LSVG: Language-Guided Scene Graphs with 2D-Assisted Multi-Modal Encoding for 3D Visual Groundingcited by 0
  8. 2025B2N3D: Progressive Learning from Binary to N-ary Relationships for 3D Object Groundingcited by 0
  9. 2026Efficient3D: A Unified Framework for Adaptive and Debiased Token Reduction in 3D MLLMscited by 0
  10. 2026SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMscited by 0
  11. 2026Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matchingcited by 0
  12. 2026CitySTAR: Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Groundingcited by 0
  13. 2026Urbex: Agentic Spatial Grounding for City-Scale 3D Scenescited by 0