# CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection

arXiv: https://arxiv.org/abs/2004.01881 · HTML: https://mlsanity.com/paper/2004.01881/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2020.

## Built on by (most cited first, 23 of 23)

- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning](https://mlsanity.com/paper/2109.06349/) (2021)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning](https://mlsanity.com/paper/2108.06332/) (2021)
- [Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System](https://mlsanity.com/paper/2104.11882/) (2021)
- [Mixture of Soft Prompts for Controllable Data Generation](https://mlsanity.com/paper/2303.01580/) (2023)
- [CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks](https://mlsanity.com/paper/2310.14623/) (2023)
- [Pseudo Siamese Network for Few-shot Intent Generation](https://mlsanity.com/paper/2105.00896/) (2021)
- [Enhancing Cross-lingual Transfer via Phonemic Transcription Integration](https://mlsanity.com/paper/2307.04361/) (2023)
- [Generate then Refine: Data Augmentation for Zero-shot Intent Detection](https://mlsanity.com/paper/2410.01953/) (2024)
- [Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languages](https://mlsanity.com/paper/2411.02398/) (2024)
- [Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots](https://mlsanity.com/paper/2203.00732/) (2022)
- [Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training](https://mlsanity.com/paper/2306.05278/) (2023)
- [Controlled Cue Generation for Play Scripts](https://mlsanity.com/paper/2112.06953/) (2021)
- [Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning](https://mlsanity.com/paper/2308.04712/) (2023)
- [CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts](https://mlsanity.com/paper/2404.12618/) (2024)
- [Synthetic Text Detection: Systemic Literature Review](https://mlsanity.com/paper/2210.06336/) (2022)
- [Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation](https://mlsanity.com/paper/2308.07645/) (2023)
- [Beyond Plain Toxic: Detection of Inappropriate Statements on Flammable Topics for the Russian Language](https://mlsanity.com/paper/2203.02392/) (2022)
- [Joint Text and Label Generation for Spoken Language Understanding](https://mlsanity.com/paper/2105.05052/) (2021)
- [Exploring the Limits of Natural Language Inference Based Setup for Few-Shot Intent Detection](https://mlsanity.com/paper/2112.07434/) (2021)
- [Data Augmentation for Improving Tail-traffic Robustness in Skill-routing for Dialogue Systems](https://mlsanity.com/paper/2306.04823/) (2023)

## Abstract (arXiv)

In this paper, we formulate a more realistic and difficult problem setup for the intent detection task in natural language understanding, namely Generalized Few-Shot Intent Detection (GFSID). GFSID aims to discriminate a joint label space consisting of both existing intents which have enough labeled data and novel intents which only have a few examples for each class. To approach this problem, we propose a novel model, Conditional Text Generation with BERT (CG-BERT). CG-BERT effectively leverages a large pre-trained language model to generate text conditioned on the intent label. By modeling the utterance distribution with variational inference, CG-BERT can generate diverse utterances for the novel intents even with only a few utterances available. Experimental results show that CG-BERT achieves state-of-the-art performance on the GFSID task with 1-shot and 5-shot settings on two real-world datasets.