mlsanityData to 7 Oct 2026

Learning In-context Learning for Named Entity Recognition

arXiv 2305.11038 ↗First posted 18 May 2023cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Named entity recognition in real-world applications suffers from the diversity of entity types, the emergence of new entity types, and the lack of high-quality annotations. To address the above problems, this paper proposes an in-context learning-based NER approach, which can effectively inject in-context NER ability into PLMs and recognize entities of novel types on-the-fly using only a few demonstrative instances. Specifically, we model PLMs as a meta-function mathcal λ_ instruction, demonstrations, text. M, and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i.e., (λ . M) (instruction, demonstrations) → F where F will be a new entity extractor, i.e., F: text → entities. To inject the above in-context NER ability into PLMs, we propose a meta-function pre-training algorithm, which pre-trains PLMs by comparing the (instruction, demonstration)-initialized extractor with a surrogate golden extractor. Experimental results on 4 few-shot NER datasets show that our method can effectively inject in-context NER ability into PLMs and significantly outperforms the PLMs+fine-tuning counterparts.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

36 of 56 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2022Transformers learn in-context by gradient descentcited by 714
  12. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
24 more
  1. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  2. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  3. 2016Neural Architectures for Named Entity Recognitioncited by 548
  4. 2021MetaICL: Learning to Learn In Contextcited by 450
  5. 2022Unified Structure Generation for Universal Information Extractioncited by 220
  6. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
  7. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  8. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  9. 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
  10. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  11. 2021Template-free Prompt Tuning for Few-shot NERcited by 73
  12. 2018Few-shot classification in Named Entity Recognition Taskcited by 64
  13. 2021CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learningcited by 64
  14. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 56
  15. 2021Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NERcited by 38
  16. 2021An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labelingcited by 26
  17. 2021Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognitioncited by 21
  18. 2022Label Semantics for Few Shot Named Entity Recognitioncited by 21
  19. 2022Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypescited by 17
  20. 2022Few-shot Named Entity Recognition with Self-describing Networkscited by 16
  21. 2021Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Frameworkcited by 15
  22. 2022QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognitioncited by 15
  23. 2022Robustness of Demonstration-based Learning Under Limited Data Scenariocited by 11
  24. 2021Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervisioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 1; 2026: 2.

7 papers.

  1. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  2. 2023Guideline Learning for In-context Information Extractioncited by 24
  3. 2023How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domaincited by 14
  4. 2024PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognitioncited by 6
  5. 2025Optimizing In-Context Learning for Efficient Full Conformal Predictioncited by 1
  6. 2026Curation and Extraction of Drug-Related Entities from Reddit Platformcited by 0
  7. 2026LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extractioncited by 0