# MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Grounding

arXiv: https://arxiv.org/abs/2403.03077 · HTML: https://mlsanity.com/paper/2403.03077/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [Multi3DRefer: Grounding Text Description to Multiple 3D Objects](https://mlsanity.com/paper/2309.05251/) (2023)
- [InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring](https://mlsanity.com/paper/2103.01128/) (2021)
- [Multi-View Transformer for 3D Visual Grounding](https://mlsanity.com/paper/2204.02174/) (2022)
- [Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds](https://mlsanity.com/paper/2112.08879/) (2021)
- [SAT: 2D Semantics Assisted Training for 3D Visual Grounding](https://mlsanity.com/paper/2105.11450/) (2021)
- [Graph-Structured Representations for Visual Question Answering](https://mlsanity.com/paper/1609.05600/) (2016)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [Multimodal Intelligence: Representation Learning, Information Fusion, and Applications](https://mlsanity.com/paper/1911.03977/) (2019)
- [TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding](https://mlsanity.com/paper/2108.02388/) (2021)
- [ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidance](https://mlsanity.com/paper/2303.16894/) (2023)
- [Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships](https://mlsanity.com/paper/1807.00119/) (2018)
- [Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Grounding](https://mlsanity.com/paper/2211.14241/) (2022)
- [Point Cloud Augmentation with Weighted Local Transformations](https://mlsanity.com/paper/2110.05379/) (2021)
- [Putting visual object recognition in context](https://mlsanity.com/paper/1911.07349/) (2019)

## Built on by (most cited first, 29 of 29)

- [SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding](https://mlsanity.com/paper/2412.04383/) (2024)
- [A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions](https://mlsanity.com/paper/2406.05785/) (2024)
- [Unifying 2D and 3D Vision-Language Understanding](https://mlsanity.com/paper/2503.10745/) (2025)
- [DenseGrounding: Improving Dense Language-Vision Semantics for Ego-Centric 3D Visual Grounding](https://mlsanity.com/paper/2505.04965/) (2025)
- [SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding](https://mlsanity.com/paper/2506.21924/) (2025)
- [Task-oriented Sequential Grounding and Navigation in 3D Scenes](https://mlsanity.com/paper/2408.04034/) (2024)
- [ViewSRD: 3D Visual Grounding via Structured Multi-View Decomposition](https://mlsanity.com/paper/2507.11261/) (2025)
- [Inpaint360GS: Efficient Object-Aware 3D Inpainting via Gaussian Splatting for 360deg Scenes](https://mlsanity.com/paper/2511.06457/) (2025)
- [3D Spatial Understanding in MLLMs: Disambiguation and Evaluation](https://mlsanity.com/paper/2412.06613/) (2024)
- [ProxyTransformation: Preshaping Point Cloud Manifold With Proxy Attention For 3D Visual Grounding](https://mlsanity.com/paper/2502.19247/) (2025)
- [Multi-Object 3D Grounding with Dynamic Modules and Language-Informed Spatial Attention](https://mlsanity.com/paper/2410.22306/) (2024)
- [From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes](https://mlsanity.com/paper/2506.04897/) (2025)
- [DEGround: An Effective Baseline for Ego-centric 3D Visual Grounding with a Homogeneous Framework](https://mlsanity.com/paper/2506.05199/) (2025)
- [GroundFlow: A Plug-in Module for Temporal Reasoning on 3D Point Cloud Sequential Grounding](https://mlsanity.com/paper/2506.21188/) (2025)
- [Seeing Clearly, Forgetting Deeply: Revisiting Fine-Tuned Video Generators for Driving Simulation](https://mlsanity.com/paper/2508.16512/) (2025)
- [PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding](https://mlsanity.com/paper/2512.20907/) (2025)
- [Zero-Shot 3D Visual Grounding from Vision-Language Models](https://mlsanity.com/paper/2505.22429/) (2025)
- [TriCLIP-3D: A Unified Parameter-Efficient Framework for Tri-Modal 3D Visual Grounding based on CLIP](https://mlsanity.com/paper/2507.14904/) (2025)
- [Z3D: Zero-Shot 3D Visual Grounding from Images](https://mlsanity.com/paper/2602.03361/) (2026)
- [Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning](https://mlsanity.com/paper/2603.09512/) (2026)
- [JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas](https://mlsanity.com/paper/2603.06168/) (2026)
- [LSVG: Language-Guided Scene Graphs with 2D-Assisted Multi-Modal Encoding for 3D Visual Grounding](https://mlsanity.com/paper/2505.04058/) (2025)
- [B2N3D: Progressive Learning from Binary to N-ary Relationships for 3D Object Grounding](https://mlsanity.com/paper/2510.10194/) (2025)
- [Invaria: Learning Scale and Density Invariance in Point Clouds via Next-Resolution Prediction](https://mlsanity.com/paper/2605.15923/) (2026)
- [AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models](https://mlsanity.com/paper/2605.25901/) (2026)
- [SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs](https://mlsanity.com/paper/2605.28490/) (2026)
- [Agentic Collaborative Cognition for Zero-Shot 3D Understanding](https://mlsanity.com/paper/2606.24649/) (2026)
- [PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding](https://mlsanity.com/paper/2606.31148/) (2026)
- [Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding](https://mlsanity.com/paper/2609.04741/) (2026)

## Abstract (arXiv)

3D visual grounding involves matching natural language descriptions with their corresponding objects in 3D spaces. Existing methods often face challenges with accuracy in object recognition and struggle in interpreting complex linguistic queries, particularly with descriptions that involve multiple anchors or are view-dependent. In response, we present the MiKASA (Multi-Key-Anchor Scene-Aware) Transformer. Our novel end-to-end trained model integrates a self-attention-based scene-aware object encoder and an original multi-key-anchor technique, enhancing object recognition accuracy and the understanding of spatial relationships. Furthermore, MiKASA improves the explainability of decision-making, facilitating error diagnosis. Our model achieves the highest overall accuracy in the Referit3D challenge for both the Sr3D and Nr3D datasets, particularly excelling by a large margin in categories that require viewpoint-dependent descriptions.