# Adaptive Self-training for Few-shot Neural Sequence Labeling

arXiv: https://arxiv.org/abs/2010.03680 · HTML: https://mlsanity.com/paper/2010.03680/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [Self-Training with Weak Supervision](https://mlsanity.com/paper/2104.05514/) (2021)
- [Multimodal Emergent Fake News Detection via Meta Neural Process Networks](https://mlsanity.com/paper/2106.13711/) (2021)
- [Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data](https://mlsanity.com/paper/2106.08977/) (2021)
- [Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Framework](https://mlsanity.com/paper/2109.05357/) (2021)
- [Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU](https://mlsanity.com/paper/2109.08259/) (2021)
- [Cluster & Tune: Boost Cold Start Performance in Text Classification](https://mlsanity.com/paper/2203.10581/) (2022)
- [Meta Self-Refinement for Robust Learning with Weak Supervision](https://mlsanity.com/paper/2205.07290/) (2022)
- [FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition](https://mlsanity.com/paper/2208.11464/) (2022)
- [PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks](https://mlsanity.com/paper/2210.08855/) (2022)
- [Learning to Detect Noisy Labels Using Model-Based Features](https://mlsanity.com/paper/2212.13767/) (2022)
- [Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining?](https://mlsanity.com/paper/2203.12881/) (2022)
- [LLM-enhanced Self-training for Cross-domain Constituency Parsing](https://mlsanity.com/paper/2311.02660/) (2023)
- [LST: Lexicon-Guided Self-Training for Few-Shot Text Classification](https://mlsanity.com/paper/2202.02566/) (2022)
- [How to choose "Good" Samples for Text Data Augmentation](https://mlsanity.com/paper/2302.00894/) (2023)
- [Friend-training: Learning from Models of Different but Related Tasks](https://mlsanity.com/paper/2301.13683/) (2023)
- [KEST: Kernel Distance Based Efficient Self-Training for Improving Controllable Text Generation](https://mlsanity.com/paper/2306.10414/) (2023)
- [Enhancing AI Research Paper Analysis: Methodology Component Extraction using Factored Transformer-based Sequence Modeling Approach](https://mlsanity.com/paper/2311.03401/) (2023)

## Abstract (arXiv)

Sequence labeling is an important technique employed for many Natural Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot tagging for dialog systems and semantic parsing. Large-scale pre-trained language models obtain very good performance on these tasks when fine-tuned on large amounts of task-specific labeled data. However, such large-scale labeled datasets are difficult to obtain for several tasks and domains due to the high cost of human annotation as well as privacy and data access constraints for sensitive user applications. This is exacerbated for sequence labeling tasks requiring such annotations at token-level. In this work, we develop techniques to address the label scarcity challenge for neural sequence labeling models. Specifically, we develop self-training and meta-learning techniques for training neural sequence taggers with few labels. While self-training serves as an effective mechanism to learn from large amounts of unlabeled data -- meta-learning helps in adaptive sample re-weighting to mitigate error propagation from noisy pseudo-labels. Extensive experiments on six benchmark datasets including two for massive multilingual NER and four slot tagging datasets for task-oriented dialog systems demonstrate the effectiveness of our method. With only 10 labeled examples for each class for each task, our method obtains 10% improvement over state-of-the-art systems demonstrating its effectiveness for the low-resource setting.