# Highly Parallel Autoregressive Entity Linking with Discriminative Correction

arXiv: https://arxiv.org/abs/2109.03792 · HTML: https://mlsanity.com/paper/2109.03792/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 58 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Entities as Experts: Sparse Memory Access with Entity Supervision](https://mlsanity.com/paper/2004.07202/) (2020)
- [End-to-End Neural Entity Linking](https://mlsanity.com/paper/1808.07699/) (2018)
- [Deep Joint Entity Disambiguation with Local Neural Attention](https://mlsanity.com/paper/1704.04920/) (2017)
- [Efficient One-Pass End-to-End Entity Linking for Questions](https://mlsanity.com/paper/2010.02413/) (2020)
- [REL: An Entity Linker Standing on the Shoulders of Giants](https://mlsanity.com/paper/2006.01969/) (2020)
- [Question Answering by Reasoning Across Documents with Graph Convolutional Networks](https://mlsanity.com/paper/1808.09920/) (2018)
- [Improving Entity Linking by Modeling Latent Relations between Mentions](https://mlsanity.com/paper/1804.10637/) (2018)
- [Entity Linking in 100 Languages](https://mlsanity.com/paper/2011.02690/) (2020)
- [NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned](https://mlsanity.com/paper/2101.00133/) (2021)
- [Revealing the Importance of Semantic Retrieval for Machine Reading at Scale](https://mlsanity.com/paper/1909.08041/) (2019)
- [Multilingual Autoregressive Entity Linking](https://mlsanity.com/paper/2103.12528/) (2021)
- [Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking](https://mlsanity.com/paper/2003.05473/) (2020)
- [Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation](https://mlsanity.com/paper/1601.01343/) (2016)
- [Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge](https://mlsanity.com/paper/2007.00849/) (2020)
- [A Memory Efficient Baseline for Open Domain Question Answering](https://mlsanity.com/paper/2012.15156/) (2020)
- [OTTers: One-turn Topic Transitions for Open-Domain Dialogue](https://mlsanity.com/paper/2105.13710/) (2021)
- [Joint Learning of Named Entity Recognition and Entity Linking](https://mlsanity.com/paper/1907.08243/) (2019)

## Built on by (most cited first, 21 of 21)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [IRGen: Generative Modeling for Image Retrieval](https://mlsanity.com/paper/2303.10126/) (2023)
- [Instructed Language Models with Retrievers Are Powerful Entity Linkers](https://mlsanity.com/paper/2311.03250/) (2023)
- [ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget](https://mlsanity.com/paper/2408.00103/) (2024)
- [TempEL: Linking Dynamically Evolving and Newly Emerging Entities](https://mlsanity.com/paper/2302.02500/) (2023)
- [OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs](https://mlsanity.com/paper/2409.05152/) (2024)
- [Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem](https://mlsanity.com/paper/2204.05990/) (2022)
- [SpEL: Structured Prediction for Entity Linking](https://mlsanity.com/paper/2310.14684/) (2023)
- [AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes](https://mlsanity.com/paper/2305.14725/) (2023)
- [Multilingual End to End Entity Linking](https://mlsanity.com/paper/2306.08896/) (2023)
- [Bidirectional End-to-End Learning of Retriever-Reader Paradigm for Entity Linking](https://mlsanity.com/paper/2306.12245/) (2023)
- [LELA: an LLM-based Entity Linking Approach with Zero-Shot Domain Adaptation](https://mlsanity.com/paper/2601.05192/) (2026)
- [Benchmarking Diverse-Modal Entity Linking with Generative Models](https://mlsanity.com/paper/2305.17337/) (2023)
- [ELEVANT: A Fully Automatic Fine-Grained Entity Linking Evaluation and Analysis Tool](https://mlsanity.com/paper/2208.07193/) (2022)
- [BELHD: Improving Biomedical Entity Linking with Homonoym Disambiguation](https://mlsanity.com/paper/2401.05125/) (2024)
- [Entity Disambiguation via Fusion Entity Decoding](https://mlsanity.com/paper/2404.01626/) (2024)
- [Unified Examination of Entity Linking in Absence of Candidate Sets](https://mlsanity.com/paper/2404.11061/) (2024)
- [A Fair and In-Depth Evaluation of Existing End-to-End Entity Linking Systems](https://mlsanity.com/paper/2305.14937/) (2023)
- [BELXTR: Biomedical Entity Linking via Contextualized Token Retrieval](https://mlsanity.com/paper/2609.25859/) (2026)

## Abstract (arXiv)

Generative approaches have been recently shown to be effective for both Entity Disambiguation and Entity Linking (i.e., joint mention detection and disambiguation). However, the previously proposed autoregressive formulation for EL suffers from i) high computational cost due to a complex (deep) decoder, ii) non-parallelizable decoding that scales with the source sequence length, and iii) the need for training on a large amount of data. In this work, we propose a very efficient approach that parallelizes autoregressive linking across all potential mentions and relies on a shallow and efficient decoder. Moreover, we augment the generative objective with an extra discriminative component, i.e., a correction term which lets us directly optimize the generator's ranking. When taken together, these techniques tackle all the above issues: our model is >70 times faster and more accurate than the previous generative method, outperforming state-of-the-art approaches on the standard English dataset AIDA-CoNLL. Source code available at https://github.com/nicola-decao/efficient-autoregressive-EL