# Learning In-context Learning for Named Entity Recognition

arXiv: https://arxiv.org/abs/2305.11038 · HTML: https://mlsanity.com/paper/2305.11038/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 56 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [What learning algorithm is in-context learning? Investigations with linear models](https://mlsanity.com/paper/2211.15661/) (2022)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network](https://mlsanity.com/paper/2006.05702/) (2020)
- [Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again](https://mlsanity.com/paper/2203.08410/) (2022)
- [Template-free Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2109.13532/) (2021)
- [Few-shot classification in Named Entity Recognition Task](https://mlsanity.com/paper/1812.06158/) (2018)
- [CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2109.07589/) (2021)
- [Improving In-Context Few-Shot Learning via Self-Supervised Training](https://mlsanity.com/paper/2205.01703/) (2022)
- [Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER](https://mlsanity.com/paper/2110.08454/) (2021)
- [An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling](https://mlsanity.com/paper/2109.13023/) (2021)
- [Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition](https://mlsanity.com/paper/2106.15167/) (2021)
- [Label Semantics for Few Shot Named Entity Recognition](https://mlsanity.com/paper/2203.08985/) (2022)

## Built on by (most cited first, 7 of 7)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Guideline Learning for In-context Information Extraction](https://mlsanity.com/paper/2310.05066/) (2023)
- [How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domain](https://mlsanity.com/paper/2307.00186/) (2023)
- [PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition](https://mlsanity.com/paper/2402.04838/) (2024)
- [Optimizing In-Context Learning for Efficient Full Conformal Prediction](https://mlsanity.com/paper/2509.01840/) (2025)
- [Curation and Extraction of Drug-Related Entities from Reddit Platform](https://mlsanity.com/paper/2605.26445/) (2026)
- [LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction](https://mlsanity.com/paper/2606.29407/) (2026)

## Abstract (arXiv)

Named entity recognition in real-world applications suffers from the diversity of entity types, the emergence of new entity types, and the lack of high-quality annotations. To address the above problems, this paper proposes an in-context learning-based NER approach, which can effectively inject in-context NER ability into PLMs and recognize entities of novel types on-the-fly using only a few demonstrative instances. Specifically, we model PLMs as a meta-function mathcal λ_ instruction, demonstrations, text. M, and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i.e.,  (λ . M) (instruction, demonstrations) → F where F will be a new entity extractor, i.e., F: text → entities. To inject the above in-context NER ability into PLMs, we propose a meta-function pre-training algorithm, which pre-trains PLMs by comparing the (instruction, demonstration)-initialized extractor with a surrogate golden extractor. Experimental results on 4 few-shot NER datasets show that our method can effectively inject in-context NER ability into PLMs and significantly outperforms the PLMs+fine-tuning counterparts.