# Robustness Evaluation of Entity Disambiguation Using Prior Probes:the Case of Entity Overshadowing

arXiv: https://arxiv.org/abs/2108.10949 · HTML: https://mlsanity.com/paper/2108.10949/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 25 references that are in the map.
- First posted on arXiv 24 Aug 2021.

## Builds on (most cited first, 4 of 4)

- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [REL: An Entity Linker Standing on the Shoulders of Giants](https://mlsanity.com/paper/2006.01969/) (2020)
- [The Clever Hans Effect in Anomaly Detection](https://mlsanity.com/paper/2006.10609/) (2020)

## Built on by (most cited first, 11 of 11)

- [ReFinED: An Efficient Zero-shot-capable Approach to End-to-End Entity Linking](https://mlsanity.com/paper/2207.04108/) (2022)
- [TempEL: Linking Dynamically Evolving and Newly Emerging Entities](https://mlsanity.com/paper/2302.02500/) (2023)
- [Improving Entity Disambiguation by Reasoning over a Knowledge Base](https://mlsanity.com/paper/2207.04106/) (2022)
- [GLADIS: A General and Large Acronym Disambiguation Benchmark](https://mlsanity.com/paper/2302.01860/) (2023)
- [Entity Disambiguation via Fusion Entity Decoding](https://mlsanity.com/paper/2404.01626/) (2024)
- [Hansel: A Chinese Few-Shot and Zero-Shot Entity Linking Benchmark](https://mlsanity.com/paper/2207.13005/) (2022)
- [REXEL: An End-to-end Model for Document-Level Relation Extraction and Entity Linking](https://mlsanity.com/paper/2404.12788/) (2024)
- [Musical Heritage Historical Entity Linking](https://mlsanity.com/paper/2502.09168/) (2025)
- [Focusing on Context is NICE: Improving Overshadowed Entity Disambiguation](https://mlsanity.com/paper/2210.06164/) (2022)
- [CADEL: A Corpus of Administrative Web Documents for Japanese Entity Linking](https://mlsanity.com/paper/2603.29336/) (2026)
- [Select, Don't Train: The Benefits of Modular Entity Disambiguation with LLM-Based Selection](https://mlsanity.com/paper/2608.27470/) (2026)

## Abstract (arXiv)

Entity disambiguation (ED) is the last step of entity linking (EL), when candidate entities are reranked according to the context they appear in. All datasets for training and evaluating models for EL consist of convenience samples, such as news articles and tweets, that propagate the prior probability bias of the entity distribution towards more frequently occurring entities. It was previously shown that the performance of the EL systems on such datasets is overestimated since it is possible to obtain higher accuracy scores by merely learning the prior. To provide a more adequate evaluation benchmark, we introduce the ShadowLink dataset, which includes 16K short text snippets annotated with entity mentions. We evaluate and report the performance of popular EL systems on the ShadowLink benchmark. The results show a considerable difference in accuracy between more and less common entities for all of the EL systems under evaluation, demonstrating the effects of prior probability bias and entity overshadowing.