TOKEN is a MASK: Few-shot Named Entity Recognition with Pre-trained Language Models
Not on the map. Why?
Abstract (arXiv)
Transferring knowledge from one domain to another is of practical importance for many tasks in natural language processing, especially when the amount of available data in the target domain is limited. In this work, we propose a novel few-shot approach to domain adaptation in the context of Named Entity Recognition (NER). We propose a two-step approach consisting of a variable base module and a template module that leverages the knowledge captured in pre-trained language models with the help of simple descriptive patterns. Our approach is simple yet versatile and can be applied in few-shot and zero-shot settings. Evaluating our lightweight approach across a number of different datasets shows that it can boost the performance of state-of-the-art baselines by 2-5% F1-score.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 15 Jun 2022.
Builds on
17 of 26 references.
References · 17
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
- 2018Deep contextualized word representationscited by 2,290
- 2019Language Models as Knowledge Bases?cited by 1,936
- 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
- 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
5 more
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
- 2018Few-shot classification in Named Entity Recognition Taskcited by 64
- 2020Few-Shot Named Entity Recognition: A Comprehensive Studycited by 41
- 2020Learning to Classify Intents and Slot Labels Given a Handful of Examplescited by 31
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.