# DRIN: Dynamic Relation Interactive Network for Multimodal Entity Linking

arXiv: https://arxiv.org/abs/2310.05589 · HTML: https://mlsanity.com/paper/2310.05589/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [End-to-End Neural Entity Linking](https://mlsanity.com/paper/1808.07699/) (2018)
- [Deep Joint Entity Disambiguation with Local Neural Attention](https://mlsanity.com/paper/1704.04920/) (2017)
- [N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification](https://mlsanity.com/paper/1802.08888/) (2018)
- [WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types](https://mlsanity.com/paper/2204.06347/) (2022)
- [Multimodal Entity Linking for Tweets](https://mlsanity.com/paper/2104.03236/) (2021)
- [Neural Collective Entity Linking](https://mlsanity.com/paper/1811.08603/) (2018)
- [Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks](https://mlsanity.com/paper/1604.00734/) (2016)
- [Multi-Grained Multimodal Interaction Network for Entity Linking](https://mlsanity.com/paper/2307.09721/) (2023)
- [Entity-aware ELMo: Learning Contextual Entity Representation for Entity Disambiguation](https://mlsanity.com/paper/1908.05762/) (2019)

## Built on by (most cited first, 8 of 8)

- [Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking](https://mlsanity.com/paper/2406.01934/) (2024)
- [UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models](https://mlsanity.com/paper/2407.16160/) (2024)
- [DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking](https://mlsanity.com/paper/2508.15876/) (2025)
- [Multi-level Matching Network for Multimodal Entity Linking](https://mlsanity.com/paper/2412.10440/) (2024)
- [OVEL: Large Language Model as Memory Manager for Online Video Entity Linking](https://mlsanity.com/paper/2403.01411/) (2024)
- [MERLIN: A Testbed for Multilingual Multimodal Entity Recognition and Linking](https://mlsanity.com/paper/2510.14307/) (2025)
- [VP-MEL: Visual Prompts Guided Multimodal Entity Linking](https://mlsanity.com/paper/2412.06720/) (2024)
- [I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking](https://mlsanity.com/paper/2508.02243/) (2025)

## Abstract (arXiv)

Multimodal Entity Linking (MEL) is a task that aims to link ambiguous mentions within multimodal contexts to referential entities in a multimodal knowledge base. Recent methods for MEL adopt a common framework: they first interact and fuse the text and image to obtain representations of the mention and entity respectively, and then compute the similarity between them to predict the correct entity. However, these methods still suffer from two limitations: first, as they fuse the features of text and image before matching, they cannot fully exploit the fine-grained alignment relations between the mention and entity. Second, their alignment is static, leading to low performance when dealing with complex and diverse data. To address these issues, we propose a novel framework called Dynamic Relation Interactive Network (DRIN) for MEL tasks. DRIN explicitly models four different types of alignment between a mention and entity and builds a dynamic Graph Convolutional Network (GCN) to dynamically select the corresponding alignment relations for different input samples. Experiments on two datasets show that DRIN outperforms state-of-the-art methods by a large margin, demonstrating the effectiveness of our approach.