# Multilingual Autoregressive Entity Linking

arXiv: https://arxiv.org/abs/2103.12528 · HTML: https://mlsanity.com/paper/2103.12528/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [Scalable Zero-shot Entity Linking with Dense Entity Retrieval](https://mlsanity.com/paper/1911.03814/) (2019)
- [Deep Joint Entity Disambiguation with Local Neural Attention](https://mlsanity.com/paper/1704.04920/) (2017)
- [Question Answering by Reasoning Across Documents with Graph Convolutional Networks](https://mlsanity.com/paper/1808.09920/) (2018)
- [Entity Linking in 100 Languages](https://mlsanity.com/paper/2011.02690/) (2020)
- [NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned](https://mlsanity.com/paper/2101.00133/) (2021)
- [Revealing the Importance of Semantic Retrieval for Machine Reading at Scale](https://mlsanity.com/paper/1909.08041/) (2019)
- [Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation](https://mlsanity.com/paper/1601.01343/) (2016)
- [Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning](https://mlsanity.com/paper/1702.03274/) (2017)
- [A Memory Efficient Baseline for Open Domain Question Answering](https://mlsanity.com/paper/2012.15156/) (2020)
- [Neural Cross-Lingual Entity Linking](https://mlsanity.com/paper/1712.01813/) (2017)
- [Joint Multilingual Supervision for Cross-lingual Entity Linking](https://mlsanity.com/paper/1809.07657/) (2018)
- [Towards Zero-resource Cross-lingual Entity Linking](https://mlsanity.com/paper/1909.13180/) (2019)

## Built on by (most cited first, 30 of 50)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities](https://mlsanity.com/paper/2302.11154/) (2023)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation](https://mlsanity.com/paper/2206.10128/) (2022)
- [One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval](https://mlsanity.com/paper/2107.11976/) (2021)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition](https://mlsanity.com/paper/2210.12391/) (2022)
- [Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning](https://mlsanity.com/paper/2204.05164/) (2022)
- [An Open Challenge for Inductive Link Prediction on Knowledge Graphs](https://mlsanity.com/paper/2203.01520/) (2022)
- [Dataset Geography: Mapping Language Data to Language Users](https://mlsanity.com/paper/2112.03497/) (2021)
- [Highly Parallel Autoregressive Entity Linking with Discriminative Correction](https://mlsanity.com/paper/2109.03792/) (2021)
- [UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQL](https://mlsanity.com/paper/2203.07781/) (2022)
- [IRGen: Generative Modeling for Image Retrieval](https://mlsanity.com/paper/2303.10126/) (2023)
- [MKGL: Mastery of a Three-Word Language](https://mlsanity.com/paper/2410.07526/) (2024)
- [TempEL: Linking Dynamically Evolving and Newly Emerging Entities](https://mlsanity.com/paper/2302.02500/) (2023)
- [A Review on Knowledge Graphs for Healthcare: Resources, Applications, and Promises](https://mlsanity.com/paper/2306.04802/) (2023)
- [Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem](https://mlsanity.com/paper/2204.05990/) (2022)
- [Large Language Models Meet Knowledge Graphs to Answer Factoid Questions](https://mlsanity.com/paper/2310.02166/) (2023)
- [MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network](https://mlsanity.com/paper/2106.07352/) (2021)
- [Improving Candidate Retrieval with Entity Profile Generation for Wikidata Entity Linking](https://mlsanity.com/paper/2202.13404/) (2022)
- [Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023?](https://mlsanity.com/paper/2212.09747/) (2022)
- [Mind the Gap: Cross-Lingual Information Retrieval with Hierarchical Knowledge Enhancement](https://mlsanity.com/paper/2112.13510/) (2021)
- [AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes](https://mlsanity.com/paper/2305.14725/) (2023)
- [Multilingual End to End Entity Linking](https://mlsanity.com/paper/2306.08896/) (2023)
- [mOKB6: A Multilingual Open Knowledge Base Completion Benchmark](https://mlsanity.com/paper/2211.06959/) (2022)
- [Multilingual Fact Linking](https://mlsanity.com/paper/2109.14364/) (2021)
- [Data-Efficient Autoregressive Document Retrieval for Fact Verification](https://mlsanity.com/paper/2211.09388/) (2022)
- [LELA: an LLM-based Entity Linking Approach with Zero-Shot Domain Adaptation](https://mlsanity.com/paper/2601.05192/) (2026)

## Abstract (arXiv)

We present mGENRE, a sequence-to-sequence system for the Multilingual Entity Linking (MEL) problem -- the task of resolving language-specific mentions to a multilingual Knowledge Base (KB). For a mention in a given language, mGENRE predicts the name of the target entity left-to-right, token-by-token in an autoregressive fashion. The autoregressive formulation allows us to effectively cross-encode mention string and entity names to capture more interactions than the standard dot product between mention and entity vectors. It also enables fast search within a large KB even for mentions that do not appear in mention tables and with no need for large-scale vector indices. While prior MEL works use a single representation for each entity, we match against entity names of as many languages as possible, which allows exploiting language connections between source input and target name. Moreover, in a zero-shot setting on languages with no training data at all, mGENRE treats the target language as a latent variable that is marginalized at prediction time. This leads to over 50% improvements in average accuracy. We show the efficacy of our approach through extensive evaluation including experiments on three popular MEL benchmarks where mGENRE establishes new state-of-the-art results. Code and pre-trained models at https://github.com/facebookresearch/GENRE.