mlsanityData to 14 Oct 2026

Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning

arXiv 2109.06349 ↗First posted 13 Sep 2021cs.CL cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

In this work, we focus on a more challenging few-shot intent detection scenario where many intents are fine-grained and semantically similar. We present a simple yet effective few-shot intent detection schema via contrastive pre-training and fine-tuning. Specifically, we first conduct self-supervised contrastive pre-training on collected intent datasets, which implicitly learns to discriminate semantically similar utterances without using any labels. We then perform few-shot intent detection together with supervised contrastive learning, which explicitly pulls utterances from the same intent closer and pushes utterances across different intents farther. Experimental results show that our proposed method achieves state-of-the-art performance on three challenging intent detection datasets under 5-shot and 10-shot settings.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 29 of its 34 references that are in the map. First posted on arXiv 13 Sep 2021.

Builds on

29 of 34 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  7. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  8. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  9. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  10. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  11. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 276
  12. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
17 more
  1. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  2. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  3. 2019Benchmarking Natural Language Understanding Services for building Conversational Agentscited by 166
  4. 2018Semantic Parsing for Task Oriented Dialog using Hierarchical Representationscited by 101
  5. 2020DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialoguecited by 89
  6. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  7. 2019Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Trackingcited by 75
  8. 2021Contrastive Self-supervised Sequential Recommendation with Robust Augmentationcited by 56
  9. 2020Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inferencecited by 47
  10. 2021Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformercited by 45
  11. 2021Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and Systemcited by 30
  12. 2020Example-Driven Intent Prediction with Observerscited by 26
  13. 2020CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detectioncited by 23
  14. 2021Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detectioncited by 22
  15. 2020Composed Variational Natural Language Generation for Few-shot Intentscited by 19
  16. 2020Basket Recommendation with Multi-Intent Translation Graph Neural Networkcited by 16
  17. 2021Pseudo Siamese Network for Few-shot Intent Generationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 15; 2023: 17; 2024: 7; 2025: 5; 2026: 2.

46 papers.

Sort:

By area:

  1. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 57
  2. 2023ConDA: Contrastive Domain Adaptation for AI-generated Text Detectioncited by 37
  3. 2022Generalized Category Discovery with Decoupled Prototypical Networkcited by 35
  4. 2022SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generationcited by 34
  5. 2022New Intent Discovery with Pre-training and Contrastive Learningcited by 28
  6. 2022SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understandingcited by 28
  7. 2023Mask-guided BERT for Few Shot Text Classificationcited by 25
  8. 2023PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillationcited by 25
  9. 2023Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Informationcited by 24
  10. 2023Exploring Zero and Few-shot Techniques for Intent Classificationcited by 21
  11. 2022Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropizationcited by 19
  12. 2022Whodunit? Learning to Contrast for Authorship Attributioncited by 17
34 more
  1. 2024Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Textcited by 11
  2. 2022Improving Fine-tuning of Self-supervised Models with Contrastive Initializationcited by 10
  3. 2022NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialoguecited by 8
  4. 2024IntentGPT: Few-shot Intent Discovery with Large Language Modelscited by 8
  5. 2022Emergent Quantized Communicationcited by 7
  6. 2023Hallucination Detection for Grounded Instruction Generationcited by 7
  7. 2023Transfer and Alignment Network for Generalized Category Discoverycited by 7
  8. 2024New Intent Discovery with Attracting and Dispersing Prototypecited by 6
  9. 2024Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classificationcited by 6
  10. 2023QAID: Question Answering Inspired Few-shot Intent Detectioncited by 5
  11. 2023Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-trainingcited by 5
  12. 2023InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systemscited by 5
  13. 2024ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Fillercited by 5
  14. 2022Learning to Select from Multiple Optionscited by 4
  15. 2024Domain Adaptation in Intent Classification Systems: A Reviewcited by 4
  16. 2022Learning Better Intent Representations for Financial Open Intent Classificationcited by 3
  17. 2023Dense Retrieval as Indirect Supervision for Large-space Decision Makingcited by 3
  18. 2022The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classificationcited by 2
  19. 2025Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptationcited by 2
  20. 2022Contrastive Learning of Sociopragmatic Meaning in Social Mediacited by 1
  21. 2023Multi-Tenant Optimization For Few-Shot Task-Oriented FAQ Retrievalcited by 1
  22. 2023All Labels Together: Low-shot Intent Detection with an Efficient Label Semantic Encoding Paradigmcited by 1
  23. 2023RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Trainingcited by 1
  24. 2023IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discoverycited by 1
  25. 2024Unleashing the Potential of Model Bias for Generalized Category Discoverycited by 1
  26. 2025Few-Shot Query Intent Detection via Relation-Aware Prompt Learningcited by 1
  27. 2022TCBERT: A Technical Report for Chinese Topic Classification BERTcited by 0
  28. 2023Multi-View Zero-Shot Open Intent Induction from Dialogues: Multi Domain Batch and Proxy Gradient Transfercited by 0
  29. 2023Information Type Classification with Contrastive Task-Specialized Sentence Encoderscited by 0
  30. 2025TARDiS : Text Augmentation for Refining Diversity and Separabilitycited by 0
  31. 2025TCProF: Time-Complexity Prediction SSL Frameworkcited by 0
  32. 2025Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detectioncited by 0
  33. 2026RuleChef: Grounding LLM Task Knowledge in Human-Editable Rulescited by 0
  34. 2026When Do LLMs Replace Fine-Tuned NLU? A Decision Framework for Intent Detection in Production Conversational Systemscited by 0