# Improving Zero-Shot Entity Retrieval through Effective Dense Representations

arXiv: https://arxiv.org/abs/2103.04156 · HTML: https://mlsanity.com/paper/2103.04156/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring](https://mlsanity.com/paper/1905.01969/) (2019)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Zero-Shot Entity Linking by Reading Entity Descriptions](https://mlsanity.com/paper/1906.07348/) (2019)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT](https://mlsanity.com/paper/1911.03681/) (2019)
- [Deep Joint Entity Disambiguation with Local Neural Attention](https://mlsanity.com/paper/1704.04920/) (2017)
- [Efficient One-Pass End-to-End Entity Linking for Questions](https://mlsanity.com/paper/2010.02413/) (2020)
- [Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation](https://mlsanity.com/paper/1601.01343/) (2016)
- [Transition-Based Dependency Parsing with Stack Long Short-Term Memory](https://mlsanity.com/paper/1505.08075/) (2015)
- [DeepType: Multilingual Entity Linking by Neural Type System Evolution](https://mlsanity.com/paper/1802.01021/) (2018)
- [Joint Learning of Named Entity Recognition and Entity Linking](https://mlsanity.com/paper/1907.08243/) (2019)
- [Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking](https://mlsanity.com/paper/1807.05127/) (2018)
- [Improving Entity Linking by Modeling Latent Entity Type Information](https://mlsanity.com/paper/2001.01447/) (2020)
- [Zero-shot Entity Linking with Efficient Long Range Sequence Modeling](https://mlsanity.com/paper/2010.06065/) (2020)
- [Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings](https://mlsanity.com/paper/2102.01156/) (2021)
- [Scalable graph-based individual named entity identification](https://mlsanity.com/paper/1811.10547/) (2018)

## Built on by (most cited first, 6 of 6)

- [MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations](https://mlsanity.com/paper/2109.05716/) (2021)
- [Modeling Fine-grained Information via Knowledge-aware Hierarchical Graph for Zero-shot Entity Retrieval](https://mlsanity.com/paper/2211.10991/) (2022)
- [NASTyLinker: NIL-Aware Scalable Transformer-based Entity Linker](https://mlsanity.com/paper/2303.04426/) (2023)
- [Towards Better Entity Linking with Multi-View Enhanced Distillation](https://mlsanity.com/paper/2305.17371/) (2023)
- [Question Answering with Texts and Tables through Deep Reinforcement Learning](https://mlsanity.com/paper/2407.04858/) (2024)
- [From Regulatory Approvals to Patents: Cross-Domain Linking for Cardiovascular Device Traceability](https://mlsanity.com/paper/2606.28353/) (2026)

## Abstract (arXiv)

Entity Linking (EL) seeks to align entity mentions in text to entries in a knowledge-base and is usually comprised of two phases: candidate generation and candidate ranking. While most methods focus on the latter, it is the candidate generation phase that sets an upper bound to both time and accuracy performance of the overall EL system. This work's contribution is a significant improvement in candidate generation which thus raises the performance threshold for EL, by generating candidates that include the gold entity in the least candidate set (top-K). We propose a simple approach that efficiently embeds mention-entity pairs in dense space through a BERT-based bi-encoder. Specifically, we extend (Wu et al., 2020) by introducing a new pooling function and incorporating entity type side-information. We achieve a new state-of-the-art 84.28% accuracy on top-50 candidates on the Zeshel dataset, compared to the previous 82.06% on the top-64 of (Wu et al., 2020). We report the results from extensive experimentation using our proposed model on both seen and unseen entity datasets. Our results suggest that our method could be a useful complement to existing EL approaches.