# PGMEL: Policy Gradient-based Generative Adversarial Network for Multimodal Entity Linking

arXiv: https://arxiv.org/abs/2510.02726 · HTML: https://mlsanity.com/paper/2510.02726/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Gated Multimodal Units for Information Fusion](https://mlsanity.com/paper/1702.01992/) (2017)
- [IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models](https://mlsanity.com/paper/1705.10513/) (2017)
- [Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation](https://mlsanity.com/paper/1601.01343/) (2016)
- [NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification](https://mlsanity.com/paper/2112.13214/) (2021)
- [Reinforced Negative Sampling over Knowledge Graph for Recommendation](https://mlsanity.com/paper/2003.05753/) (2020)
- [Understanding Hard Negatives in Noise Contrastive Estimation](https://mlsanity.com/paper/2104.06245/) (2021)
- [WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types](https://mlsanity.com/paper/2204.06347/) (2022)
- [Incorporating GAN for Negative Sampling in Knowledge Representation Learning](https://mlsanity.com/paper/1809.11017/) (2018)
- [Multimodal Entity Linking for Tweets](https://mlsanity.com/paper/2104.03236/) (2021)
- [Entity Linking Meets Deep Learning: Techniques and Solutions](https://mlsanity.com/paper/2109.12520/) (2021)
- [Neural Collective Entity Linking](https://mlsanity.com/paper/1811.08603/) (2018)
- [Robust Deep Multi-modal Learning Based on Gated Information Fusion Network](https://mlsanity.com/paper/1807.06233/) (2018)
- [Improving Entity Linking by Modeling Latent Entity Type Information](https://mlsanity.com/paper/2001.01447/) (2020)
- [Multi-Grained Multimodal Interaction Network for Entity Linking](https://mlsanity.com/paper/2307.09721/) (2023)
- [Generative Multimodal Entity Linking](https://mlsanity.com/paper/2306.12725/) (2023)
- [Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding](https://mlsanity.com/paper/2304.11618/) (2023)
- [Adversarial Training for Community Question Answer Selection Based on Multi-scale Matching](https://mlsanity.com/paper/1804.08058/) (2018)

## Abstract (arXiv)

The task of entity linking, which involves associating mentions with their respective entities in a knowledge graph, has received significant attention due to its numerous potential applications. Recently, various multimodal entity linking (MEL) techniques have been proposed, targeted to learn comprehensive embeddings by leveraging both text and vision modalities. The selection of high-quality negative samples can potentially play a crucial role in metric/representation learning. However, to the best of our knowledge, this possibility remains unexplored in existing literature within the framework of MEL. To fill this gap, we address the multimodal entity linking problem in a generative adversarial setting where the generator is responsible for generating high-quality negative samples, and the discriminator is assigned the responsibility for the metric learning tasks. Since the generator is involved in generating samples, which is a discrete process, we optimize it using policy gradient techniques and propose a policy gradient-based generative adversarial network for multimodal entity linking (PGMEL). Experimental results based on Wiki-MEL, Richpedia-MEL and WikiDiverse datasets demonstrate that PGMEL learns meaningful representation by selecting challenging negative samples and outperforms state-of-the-art methods.