# Entity Labels Are Not Entity Signals: A Framework for Observable Relevance in Document Re-Ranking

arXiv: https://arxiv.org/abs/2606.15998 · HTML: https://mlsanity.com/paper/2606.15998/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Builds on (most cited first, 17 of 17)

- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [Neural Ranking Models with Weak Supervision](https://mlsanity.com/paper/1704.08803/) (2017)
- [Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval](https://mlsanity.com/paper/1805.07591/) (2018)
- [Query Expansion with Locally-Trained Word Embeddings](https://mlsanity.com/paper/1605.07891/) (2016)
- [Word-Entity Duet Representations for Document Ranking](https://mlsanity.com/paper/1706.06636/) (2017)
- [Using Word Embeddings for Automatic Query Expansion](https://mlsanity.com/paper/1606.07608/) (2016)
- [Graph-Embedding Empowered Entity Retrieval](https://mlsanity.com/paper/2005.02843/) (2020)
- [DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities](https://mlsanity.com/paper/2410.07722/) (2024)
- [Deep Neural Networks for Query Expansion using Word Embeddings](https://mlsanity.com/paper/1811.03514/) (2018)
- [CODEC: Complex Document and Entity Collection](https://mlsanity.com/paper/2205.04546/) (2022)
- [DREQ: Document Re-Ranking Using Entity-based Query Understanding](https://mlsanity.com/paper/2401.05939/) (2024)
- [QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking](https://mlsanity.com/paper/2510.11589/) (2025)
- [Entities as Retrieval Signals: A Systematic Study of Coverage, Supervision, and Evaluation in Entity-Oriented Ranking](https://mlsanity.com/paper/2604.05204/) (2026)

## Built on by (most cited first, 1 of 1)

- [Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search](https://mlsanity.com/paper/2607.15253/) (2026)

## Abstract (arXiv)

Entity-aware document retrieval uses query-associated entities as ranking signals, assuming that semantically relevant entities are also useful retrieval signals. We show this assumption is insufficient- and explain why. Unlike terms, which are ground-truth observations, entity links are hypotheses produced by an imperfect linker: an entity can be topically central yet provide no discriminative signal if the linker fires indiscriminately across relevant and non-relevant documents. We formalize this as a distinction between Conceptual Entity Relevance (CER)- whether an entity is topically related to a query- and Observable Entity Relevance (OER)- whether its observed presence in a collection discriminates relevant from non-relevant documents. Across four collections and annotation sources including human entity judgments, CER and OER exhibit near-chance agreement (kappa ≈ 0), while OER operationalizations agree substantially (kappa ≈ 0.5), confirming CER as the systematic outlier. CER-based supervision selects topically plausible but weakly discriminative entities, pruning fewer than 4% of non-relevant documents on some collections. Aligning supervision with OER improves non-relevant pruning by up to 10x and open-world MAP by 0.051 over BM25. Our findings motivate a shift from conceptual to observable notions of entity relevance in entity-aware retrieval.