mlsanityData to 14 Oct 2026

Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Grounding

arXiv 2211.14241 ↗First posted 25 Nov 2022cs.CVcited by 41

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

The 3D visual grounding task has been explored with visual and language streams comprehending referential language to identify target objects in 3D scenes. However, most existing methods devote the visual stream to capturing the 3D visual clues using off-the-shelf point clouds encoders. The main question we address in this paper is "can we consolidate the 3D visual stream by 2D clues synthesized from point clouds and efficiently utilize them in training and testing?". The main idea is to assist the 3D encoder by incorporating rich 2D object representations without requiring extra 2D inputs. To this end, we leverage 2D clues, synthetically generated from 3D point clouds, and empirically show their aptitude to boost the quality of the learned visual representations. We validate our approach through comprehensive experiments on Nr3D, Sr3D, and ScanRefer datasets and show consistent performance gains compared to existing methods. Our proposed module, dubbed as Look Around and Refer (LAR), significantly outperforms the state-of-the-art 3D visual grounding techniques on three benchmarks, i.e., Nr3D, Sr3D, and ScanRefer. The code is available at https://eslambakr.github.io/LAR.github.io/.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 45 of its 59 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

45 of 59 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2015VQA: Visual Question Answeringcited by 2,506
  8. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  9. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  10. 2016Modeling Context in Referring Expressionscited by 1,063
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  12. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
33 more
  1. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  2. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  3. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  4. 2018Albumentations: fast and flexible image augmentationscited by 560
  5. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  6. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  7. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  8. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  9. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  10. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  11. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 218
  12. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 217
  13. 2020Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructionscited by 208
  14. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  15. 2019Visual Semantic Reasoning for Image-Text Matchingcited by 185
  16. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  17. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
  18. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  19. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 138
  20. 2015Natural Language Object Retrievalcited by 124
  21. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  22. 2017Learning Two-Branch Neural Networks for Image-Text Matching Taskscited by 119
  23. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  24. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  25. 2020TAP: Text-Aware Pre-training for Text-VQA and Text-Captioncited by 95
  26. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 89
  27. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  28. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  29. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  30. 2016A Joint Speaker-Listener-Reinforcer Model for Referring Expressionscited by 74
  31. 2019Learning Cross-modal Context Graph for Visual Groundingcited by 49
  32. 2017Weakly-supervised Visual Grounding of Phrases with Linguistic Structurescited by 40
  33. 2016Faster R-CNN Features for Instance Searchcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 11; 2024: 16; 2025: 11; 2026: 2.

41 papers.

Sort:

By area:

  1. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  3. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  4. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 135
  5. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 133
  6. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 109
  7. 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
  8. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  9. 2024SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Groundingcited by 67
  10. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  11. 2024MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Groundingcited by 29
  12. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 25
29 more
  1. 2024RefMask3D: Language-Guided Transformer for 3D Referring Segmentationcited by 25
  2. 20243D Vision and Language Pretraining with Large-Scale Synthetic Datacited by 21
  3. 2024Multi-branch Collaborative Learning Network for 3D Visual Groundingcited by 20
  4. 2023Dense Object Grounding in 3D Scenescited by 15
  5. 2023CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Groundingcited by 13
  6. 2023ScanERU: Interactive 3D Visual Grounding based on Embodied Reference Understandingcited by 12
  7. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
  8. 20253EED: Ground Everything Everywhere in 3Dcited by 12
  9. 2023Cross3DVG: Cross-Dataset 3D Visual Grounding on Different RGB-D Scanscited by 10
  10. 2023Uni3DL: Unified Model for 3D and Language Understandingcited by 9
  11. 2024Naturally Supervised 3D Visual Grounding with Language-Regularized Concept Learnerscited by 8
  12. 2025Ges3ViG: Incorporating Pointing Gestures into Language-Based 3D Visual Grounding for Embodied Reference Understandingcited by 8
  13. 2024Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understandingcited by 7
  14. 2024SeCG: Semantic-Enhanced 3D Visual Grounding via Cross-modal Graph Attentioncited by 4
  15. 2024Multi-Object 3D Grounding with Dynamic Modules and Language-Informed Spatial Attentioncited by 4
  16. 2023Weakly-Supervised 3D Visual Grounding based on Visual Language Alignmentcited by 3
  17. 2024Data-Efficient 3D Visual Grounding via Order-Aware Referringcited by 3
  18. 2025PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Groundingcited by 3
  19. 2024R2G: Reasoning to Ground in 3D Scenescited by 2
  20. 2025Zero-Shot 3D Visual Grounding from Vision-Language Modelscited by 2
  21. 2025FreeQ-Graph: Free-form Querying with Semantic Consistent Scene Graph for 3D Scene Understandingcited by 2
  22. 2025TriCLIP-3D: A Unified Parameter-Efficient Framework for Tri-Modal 3D Visual Grounding based on CLIPcited by 2
  23. 2024Fine-Grained Spatial and Verbal Losses for 3D Visual Groundingcited by 1
  24. 2025DSM: Constructing a Diverse Semantic Map for 3D Visual Groundingcited by 0
  25. 2025Multi-Object Grounding via Hierarchical Contrastive Siamese Transformerscited by 0
  26. 2025LSVG: Language-Guided Scene Graphs with 2D-Assisted Multi-Modal Encoding for 3D Visual Groundingcited by 0
  27. 2025B2N3D: Progressive Learning from Binary to N-ary Relationships for 3D Object Groundingcited by 0
  28. 2026PC-CrossDiff: Point-Cluster Dual-Level Cross-Modal Differential Attention for Unified 3D Referring and Segmentationcited by 0
  29. 2026PruneGround: Plug-and-play Spatial Pruning for 3D Visual Groundingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning Matters for 3D Visual Grounding
  2. 2026GuideGround: VLM-guided Semantic Understanding and Viewpoint-aware Reasoning for 3D Visual Grounding
  3. 2025View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs
  4. 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
  5. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
  6. 2025From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
  7. 2025From Thousands to Billions: 3D Visual Language Grounding via Render-Supervised Distillation from 2D VLMs
  8. 2026TDVR: Joint Text Disambiguation and Viewpoint Reasoning for Zero-Shot 3D Visual Grounding