# KGMEL: Knowledge Graph-Enhanced Multimodal Entity Linking

arXiv: https://arxiv.org/abs/2504.15135 · HTML: https://mlsanity.com/paper/2504.15135/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 21 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [An Empirical Study of Training End-to-End Vision-and-Language Transformers](https://mlsanity.com/paper/2111.02387/) (2021)
- [Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader](https://mlsanity.com/paper/1905.07098/) (2019)
- [WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types](https://mlsanity.com/paper/2204.06347/) (2022)
- [Multimodal Entity Linking for Tweets](https://mlsanity.com/paper/2104.03236/) (2021)
- [Multi-Grained Multimodal Interaction Network for Entity Linking](https://mlsanity.com/paper/2307.09721/) (2023)
- [A Dual-way Enhanced Framework from Text Matching Point of View for Multimodal Entity Linking](https://mlsanity.com/paper/2312.11816/) (2023)
- [Benchmarking Diverse-Modal Entity Linking with Generative Models](https://mlsanity.com/paper/2305.17337/) (2023)
- [Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking](https://mlsanity.com/paper/2406.01934/) (2024)
- [Multi-level Matching Network for Multimodal Entity Linking](https://mlsanity.com/paper/2412.10440/) (2024)

## Built on by (most cited first, 4 of 4)

- [DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking](https://mlsanity.com/paper/2508.15876/) (2025)
- [When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs](https://mlsanity.com/paper/2604.02778/) (2026)
- [Dual-Path LLM Reasoning for Multimodal Few-Shot Knowledge Graph Completion](https://mlsanity.com/paper/2607.26909/) (2026)
- [KARAMEL: Knowledge-Aware Ranking for Multimodal Entity Linking in the Arts via Heterogeneous Graphs](https://mlsanity.com/paper/2610.16502/) (2026)

## Abstract (arXiv)

Entity linking (EL) aligns textual mentions with their corresponding entities in a knowledge base, facilitating various applications such as semantic search and question answering. Recent advances in multimodal entity linking (MEL) have shown that combining text and images can reduce ambiguity and improve alignment accuracy. However, most existing MEL methods overlook the rich structural information available in the form of knowledge-graph (KG) triples. In this paper, we propose KGMEL, a novel framework that leverages KG triples to enhance MEL. Specifically, it operates in three stages: (1) Generation: Produces high-quality triples for each mention by employing vision-language models based on its text and images. (2) Retrieval: Learns joint mention-entity representations, via contrastive learning, that integrate text, images, and (generated or KG) triples to retrieve candidate entities for each mention. (3) Reranking: Refines the KG triples of the candidate entities and employs large language models to identify the best-matching entity for the mention. Extensive experiments on benchmark datasets demonstrate that KGMEL outperforms existing methods. Our code and datasets are available at: https://github.com/juyeonnn/KGMEL.