mlsanityData to 7 Oct 2026

Multi-View Transformer for 3D Visual Grounding

arXiv 2204.02174 ↗First posted 5 Apr 2022cs.CV cs.CLcited by 138

Not on the map. Why?

Cited by 28 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The 3D visual grounding task aims to ground a natural language description to the targeted object in a 3D scene, which is usually represented in 3D point clouds. Previous works studied visual grounding under specific views. The vision-language correspondence learned by this way can easily fail once the view changes. In this paper, we propose a Multi-View Transformer (MVT) for 3D visual grounding. We project the 3D scene to a multi-view space, in which the position information of the 3D scene under different views are modeled simultaneously and aggregated together. The multi-view space enables the network to learn a more robust multi-modal representation for 3D visual grounding and eliminates the dependence on specific views. Extensive experiments show that our approach significantly outperforms all state-of-the-art methods. Specifically, on Nr3D and Sr3D datasets, our method outperforms the best competitor by 11.2% and 7.1% and even surpasses recent work with extra 2D assistance by 5.9% and 6.6%. Our code is available at https://github.com/sega-hsj/MVT-3DVG.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

33 of 39 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2016Layer Normalizationcited by 5,579
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  10. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  11. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  12. 2016Modeling Context in Referring Expressionscited by 1,054
21 more
  1. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  2. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  3. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  4. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  5. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  6. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  7. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  8. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  9. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  10. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  11. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  12. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  13. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 139
  14. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  15. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  16. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  17. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  18. 2020Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillationcited by 51
  19. 2017Self-supervised learning of visual features through embedding images into text topic spacescited by 43
  20. 2021CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptionscited by 26
  21. 2020AttnGrounder: Talking to Cars with Attentioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 28; 2024: 37; 2025: 48; 2026: 19.

138 papers.

Sort:

By area:

  1. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  2. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  3. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  4. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  5. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  6. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  7. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  8. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  9. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  10. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
  11. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  12. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
38 more
  1. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  2. 2024Grounded 3D-LLM with Referent Tokenscited by 111
  3. 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 111
  4. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  5. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  6. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  7. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 67
  8. 2024SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Groundingcited by 66
  9. 2023Visual Programming for Zero-shot Open-Vocabulary 3D Visual Groundingcited by 64
  10. 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understandingcited by 63
  11. 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awarenesscited by 61
  12. 2023NS3D: Neuro-Symbolic Grounding of 3D Objects and Relationscited by 57
  13. 2024VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Groundingcited by 56
  14. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 53
  15. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  16. 2024ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilitiescited by 47
  17. 2023What's Left? Concept Grounding with Logic-Enhanced Foundation Modelscited by 33
  18. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  19. 2023Four Ways to Improve Verbo-visual Fusion for Dense 3D Visual Groundingcited by 32
  20. 2024MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Groundingcited by 29
  21. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understandingcited by 26
  22. 2023Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understandingcited by 25
  23. 20233DRP-Net: 3D Relative Position-aware Network for 3D Visual Groundingcited by 25
  24. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 24
  25. 2023Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Cluttercited by 22
  26. 2023Mono3DVG: 3D Visual Grounding in Monocular Imagescited by 22
  27. 2025Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysiscited by 22
  28. 20243D Vision and Language Pretraining with Large-Scale Synthetic Datacited by 21
  29. 2023Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Groundingcited by 18
  30. 2024Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehensioncited by 17
  31. 2025TSP3D: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Groundingcited by 17
  32. 2022Toward Explainable and Fine-Grained 3D Grounding through Referring Textual Phrasescited by 16
  33. 2023Dense Object Grounding in 3D Scenescited by 15
  34. 2023CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Datacited by 15
  35. 2024"Where am I?" Scene Retrieval with Languagecited by 15
  36. 2023CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Groundingcited by 13
  37. 2023ScanERU: Interactive 3D Visual Grounding based on Embodied Reference Understandingcited by 12
  38. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12

50 most cited of 138 All 138 (JSON)