mlsanityData to 14 Oct 2026

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

arXiv 2609.10745 ↗First posted 9 Sep 2026cs.CLcited by 0

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected. These metrics identify many rare entities that popularity metrics miss. Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes. To address these failures, we introduce a simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically. Controlled experiments show that reasoning and retrieval are complementary. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning improves rare-entity accuracy but can hurt overall accuracy. Their combination performs best. On MERLIN, a multilingual multimodal entity linking benchmark over five languages (Hindi, Indonesian, Japanese, Tamil, Vietnamese), our best system improves over the state of the art by 6.9% overall and by up to 23.3% on rare-entity slices. We release MERLIN-Rare, rare-entity test slices for targeted evaluation, with our framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 48 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

26 of 48 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 24

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  7. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  8. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  9. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  10. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  11. 2020Autoregressive Entity Retrievalcited by 334
  12. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
12 more
  1. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  2. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 88
  3. 2021Multilingual Autoregressive Entity Linkingcited by 50
  4. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 20
  5. 2024LLMAEL: Large Language Models are Good Context Augmenters for Entity Linkingcited by 19
  6. 2025Grounding Multilingual Multimodal LLMs With Cultural Knowledgecited by 16
  7. 2024OneNet: A Fine-Tuning Free Framework for Few-Shot Entity Linking via Large Language Model Promptingcited by 9
  8. 2024EntGPT: Entity Linking with Generative Large Language Modelscited by 7
  9. 2026LELA: an LLM-based Entity Linking Approach with Zero-Shot Domain Adaptationcited by 6
  10. 2025Evaluation of LLMs on Long-tail Entity Linking in Historical Documentscited by 2
  11. 2025An Entity Linking Agent for Question Answeringcited by 2
  12. 2025MERLIN: A Testbed for Multilingual Multimodal Entity Recognition and Linkingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models
  2. 2025KGMEL: Knowledge Graph-Enhanced Multimodal Entity Linking
  3. 2026Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linking
  4. 2026TailNLG: A Multilingual Benchmark Addressing Verbalization of Long-Tail Entities
  5. 2026KARAMEL: Knowledge-Aware Ranking for Multimodal Entity Linking in the Arts via Heterogeneous Graphs
  6. 2024Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach
  7. 2025FiNERweb: Datasets and Artifacts for Scalable Multilingual Named Entity Recognition
  8. 2022Adapters for Enhanced Modeling of Multilingual Knowledge and Text