# NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval

arXiv: https://arxiv.org/abs/2310.14282 · HTML: https://mlsanity.com/paper/2310.14282/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 22 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Few-NERD: A Few-Shot Named Entity Recognition Dataset](https://mlsanity.com/paper/2105.07464/) (2021)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Ultra-Fine Entity Typing](https://mlsanity.com/paper/1807.04905/) (2018)
- [Template-free Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2109.13532/) (2021)
- [Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study](https://mlsanity.com/paper/2001.03844/) (2020)
- [Learning from Noisy Labels for Entity-Centric Information Extraction](https://mlsanity.com/paper/2104.08656/) (2021)
- [InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER](https://mlsanity.com/paper/2203.03903/) (2022)
- [QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations](https://mlsanity.com/paper/2305.11694/) (2023)
- [RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering](https://mlsanity.com/paper/2210.14353/) (2022)
- [Entity Extraction from Wikipedia List Pages](https://mlsanity.com/paper/2003.05146/) (2020)

## Built on by (most cited first, 10 of 10)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition](https://mlsanity.com/paper/2406.11192/) (2024)
- [DynamicNER: A Dynamic, Multilingual, and Fine-Grained Dataset for LLM-based Named Entity Recognition](https://mlsanity.com/paper/2409.11022/) (2024)
- [Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval](https://mlsanity.com/paper/2511.02770/) (2025)
- [Familiarity: Better Evaluation of Zero-Shot Named Entity Recognition by Quantifying Label Shifts in Synthetic Training Data](https://mlsanity.com/paper/2412.10121/) (2024)
- [NER Retriever: Zero-Shot Named Entity Retrieval with Type-Aware Embeddings](https://mlsanity.com/paper/2509.04011/) (2025)
- [FewTopNER: Integrating Few-Shot Learning with Topic Modeling and Named Entity Recognition in a Multilingual Framework](https://mlsanity.com/paper/2502.02391/) (2025)
- [Beyond Text: Characterizing Domain Expert Needs in Document Research](https://mlsanity.com/paper/2504.12495/) (2025)
- [RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering](https://mlsanity.com/paper/2602.18425/) (2026)
- [Pad=artha: Ontology-Grounded Fine-Grained NER Benchmark for Classical Sanskrit](https://mlsanity.com/paper/2608.29324/) (2026)

## Abstract (arXiv)

Recognizing entities in texts is a central need in many information-seeking scenarios, and indeed, Named Entity Recognition (NER) is arguably one of the most successful examples of a widely adopted NLP task and corresponding NLP technology. Recent advances in large language models (LLMs) appear to provide effective solutions (also) for NER tasks that were traditionally handled with dedicated models, often matching or surpassing the abilities of the dedicated models. Should NER be considered a solved problem? We argue to the contrary: the capabilities provided by LLMs are not the end of NER research, but rather an exciting beginning. They allow taking NER to the next level, tackling increasingly more useful, and increasingly more challenging, variants. We present three variants of the NER task, together with a dataset to support them. The first is a move towards more fine-grained -- and intersectional -- entity types. The second is a move towards zero-shot recognition and extraction of these fine-grained types based on entity-type labels. The third, and most challenging, is the move from the recognition setup to a novel retrieval setup, where the query is a zero-shot entity type, and the expected result is all the sentences from a large, pre-indexed corpus that contain entities of these types, and their corresponding spans. We show that all of these are far from being solved. We provide a large, silver-annotated corpus of 4 million paragraphs covering 500 entity types, to facilitate research towards all of these three goals.