# Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System

arXiv: https://arxiv.org/abs/2104.11882 · HTML: https://mlsanity.com/paper/2104.11882/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 24 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Text Classification Algorithms: A Survey](https://mlsanity.com/paper/1904.08067/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Incremental Few-Shot Learning with Attention Attractor Networks](https://mlsanity.com/paper/1810.07218/) (2018)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals](https://mlsanity.com/paper/1911.10422/) (2019)
- [Diverse Few-Shot Text Classification with Multiple Metrics](https://mlsanity.com/paper/1805.07513/) (2018)
- [Zero-shot User Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1809.00385/) (2018)
- [Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference](https://mlsanity.com/paper/2010.13009/) (2020)
- [Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start](https://mlsanity.com/paper/2010.02584/) (2020)
- [CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection](https://mlsanity.com/paper/2004.01881/) (2020)
- [Composed Variational Natural Language Generation for Few-shot Intents](https://mlsanity.com/paper/2009.10056/) (2020)
- [Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection](https://mlsanity.com/paper/2010.02481/) (2020)

## Built on by (most cited first, 30 of 30)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Meta-learning via Language Model In-context Tuning](https://mlsanity.com/paper/2110.07814/) (2021)
- [Continual Learning of Natural Language Processing Tasks: A Survey](https://mlsanity.com/paper/2211.12701/) (2022)
- [Towards Lifelong Learning of Large Language Models: A Survey](https://mlsanity.com/paper/2406.06391/) (2024)
- [Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning](https://mlsanity.com/paper/2109.06349/) (2021)
- [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://mlsanity.com/paper/2405.18653/) (2024)
- [Memory Efficient Continual Learning with Transformers](https://mlsanity.com/paper/2203.04640/) (2022)
- [Continual Training of Language Models for Few-Shot Learning](https://mlsanity.com/paper/2210.05549/) (2022)
- [Unsupervised Out-of-Domain Detection via Pre-trained Transformers](https://mlsanity.com/paper/2106.00948/) (2021)
- [Large Language Model Instruction Following: A Survey of Progresses and Challenges](https://mlsanity.com/paper/2303.10475/) (2023)
- [ConTinTin: Continual Learning from Task Instructions](https://mlsanity.com/paper/2203.08512/) (2022)
- [Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization](https://mlsanity.com/paper/2205.07208/) (2022)
- [Pseudo Siamese Network for Few-shot Intent Generation](https://mlsanity.com/paper/2105.00896/) (2021)
- [On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing](https://mlsanity.com/paper/2412.17242/) (2024)
- [What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories](https://mlsanity.com/paper/2302.03353/) (2023)
- [Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages](https://mlsanity.com/paper/2302.09685/) (2023)
- [Major Entity Identification: A Generalizable Alternative to Coreference Resolution](https://mlsanity.com/paper/2406.14654/) (2024)
- [OpenStance: Real-world Zero-shot Stance Detection](https://mlsanity.com/paper/2210.14299/) (2022)
- [Text Takes Over: A Study of Modality Bias in Multimodal Intent Detection](https://mlsanity.com/paper/2508.16122/) (2025)
- [AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs](https://mlsanity.com/paper/2511.05549/) (2025)
- [Intent Detection and Entity Extraction from BioMedical Literature](https://mlsanity.com/paper/2404.03598/) (2024)
- [Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training](https://mlsanity.com/paper/2306.05278/) (2023)
- [Learning to Select from Multiple Options](https://mlsanity.com/paper/2212.00301/) (2022)
- [Domain Adaptation in Intent Classification Systems: A Review](https://mlsanity.com/paper/2404.14415/) (2024)
- [A Pointer Network-based Approach for Joint Extraction and Detection of Multi-Label Multi-Class Intents](https://mlsanity.com/paper/2410.22476/) (2024)
- [A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection](https://mlsanity.com/paper/2205.02005/) (2022)
- [All Labels Together: Low-shot Intent Detection with an Efficient Label Semantic Encoding Paradigm](https://mlsanity.com/paper/2309.03563/) (2023)
- [Few-shot Incremental Event Detection](https://mlsanity.com/paper/2209.01979/) (2022)
- [ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining](https://mlsanity.com/paper/2210.07587/) (2022)
- [IDALC: A Semi-Supervised Framework for Intent Detection and Active Learning based Correction](https://mlsanity.com/paper/2511.05921/) (2025)

## Abstract (arXiv)

Text classification is usually studied by labeling natural language texts with relevant categories from a predefined set. In the real world, new classes might keep challenging the existing system with limited labeled data. The system should be intelligent enough to recognize upcoming new classes with a few examples. In this work, we define a new task in the NLP domain, incremental few-shot text classification, where the system incrementally handles multiple rounds of new classes. For each round, there is a batch of new classes with a few labeled examples per class. Two major challenges exist in this new task: (i) For the learning process, the system should incrementally learn new classes round by round without re-training on the examples of preceding classes; (ii) For the performance, the system should perform well on new classes without much loss on preceding classes. In addition to formulating the new task, we also release two benchmark datasets in the incremental few-shot setting: intent classification and relation classification. Moreover, we propose two entailment approaches, ENTAILMENT and HYBRID, which show promise for solving this novel problem.