mlsanityData to 7 Oct 2026

ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language

arXiv 1912.08830 ↗First posted 18 Dec 2019cs.CV cs.CL cs.LGcited by 446

Not on the map. Why?

Cited by 105 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce the task of 3D object localization in RGB-D scans using natural language descriptions. As input, we assume a point cloud of a scanned 3D scene along with a free-form description of a specified target object. To address this task, we propose ScanRefer, learning a fused descriptor from 3D object proposals and encoded sentence embeddings. This fused descriptor correlates language expressions with geometric features, enabling regression of the 3D bounding box of a target object. We also introduce the ScanRefer dataset, containing 51,583 descriptions of 11,046 objects from 800 ScanNet scenes. ScanRefer is the first large-scale effort to perform object localization via natural language expression directly in 3D.

Key facts. Cited by 446 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 34; 2023: 65; 2024: 94; 2025: 139; 2026: 97.

446 papers.

Sort:

By area:

  1. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  2. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  3. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  4. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  5. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  6. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  7. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  8. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  9. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  10. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 206
  11. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  12. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
38 more
  1. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  3. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  4. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  5. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  6. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 164
  7. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 157
  8. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 156
  9. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  10. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  11. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  12. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  13. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  14. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 133
  15. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  16. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
  17. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  18. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  19. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
  20. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  21. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  22. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  23. 2024Grounded 3D-LLM with Referent Tokenscited by 111
  24. 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 111
  25. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 109
  26. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  27. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 98
  28. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
  29. 2023LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understandingcited by 95
  30. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  31. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 88
  32. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  33. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  34. 2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 77
  35. 2024GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Graspingcited by 77
  36. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  37. 2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
  38. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70

50 most cited of 446 All 446 (JSON)