# Comprehension of Multilingual Expressions Referring to Target Objects in Visual Inputs

arXiv: https://arxiv.org/abs/2511.11427 · HTML: https://mlsanity.com/paper/2511.11427/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 39 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework](https://mlsanity.com/paper/2202.03052/) (2022)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [TransVG: End-to-End Visual Grounding with Transformers](https://mlsanity.com/paper/2104.08541/) (2021)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Universal Instance Perception as Object Discovery and Retrieval](https://mlsanity.com/paper/2303.06674/) (2023)
- [Language as Queries for Referring Video Object Segmentation](https://mlsanity.com/paper/2201.00487/) (2022)
- [Stochastic Optimization of Sorting Networks via Continuous Relaxations](https://mlsanity.com/paper/1903.08850/) (2019)
- [GREC: Generalized Referring Expression Comprehension](https://mlsanity.com/paper/2308.16182/) (2023)
- [Anchor DETR: Query Design for Transformer-Based Object Detection](https://mlsanity.com/paper/2109.07107/) (2021)
- [The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understanding](https://mlsanity.com/paper/2311.17518/) (2023)
- [FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension](https://mlsanity.com/paper/2409.14750/) (2024)
- [RefDrone: A Challenging Benchmark for Referring Expression Comprehension in Drone Scenes](https://mlsanity.com/paper/2502.00392/) (2025)
- [Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation](https://mlsanity.com/paper/1907.08948/) (2019)
- [Toloka Visual Question Answering Benchmark](https://mlsanity.com/paper/2309.16511/) (2023)
- [Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation](https://mlsanity.com/paper/2504.16060/) (2025)
- [Champion Solution for the WSDM2023 Toloka VQA Challenge](https://mlsanity.com/paper/2301.09045/) (2023)

## Built on by (most cited first, 1 of 1)

- [Scaling Representation Diversity: Modulated Attention and Reconstructive Regularization for Visual Grounding](https://mlsanity.com/paper/2608.12748/) (2026)

## Abstract (arXiv)

Referring Expression Comprehension (REC) requires models to localize objects in images based on different types of natural language descriptions. Even with significant progress, research on the area remains predominantly English-centric, despite increasing global deployment demands. This work addresses multilingual REC through two main contributions. First, we construct a unified multilingual dataset spanning 10 languages, by systematically expanding 12 existing English REC benchmarks through machine translation and context-based translation enhancement. Second, we introduce an attention-anchored efficient neural architecture that uses a multilingual SigLIP2 encoder. Our attention-based approach generates coarse spatial anchors from attention distributions, which are subsequently refined through learned residuals. Experimental evaluation demonstrates competitive performance on standard benchmarks despite the use of a relatively small model. Multilingual evaluation shows consistent capabilities across languages, establishing the practical feasibility of efficient multilingual visual grounding systems.