# ContrastNet: A Contrastive Learning Framework for Few-Shot Text Classification

arXiv: https://arxiv.org/abs/2305.09269 · HTML: https://mlsanity.com/paper/2305.09269/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 16 May 2023.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Cross Attention Network for Few-shot Classification](https://mlsanity.com/paper/1910.07677/) (2019)
- [Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation](https://mlsanity.com/paper/2001.08735/) (2020)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue](https://mlsanity.com/paper/2009.13570/) (2020)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Diverse Few-Shot Text Classification with Multiple Metrics](https://mlsanity.com/paper/1805.07513/) (2018)
- [Few-shot Text Classification with Distributional Signatures](https://mlsanity.com/paper/1908.06039/) (2019)
- [Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks](https://mlsanity.com/paper/1911.03863/) (2019)
- [Joint Contrastive Learning with Infinite Possibilities](https://mlsanity.com/paper/2009.14776/) (2020)
- [Dynamic Memory Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/2005.05727/) (2020)
- [ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning](https://mlsanity.com/paper/2105.12995/) (2021)
- [Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification](https://mlsanity.com/paper/2107.12262/) (2021)
- [Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning](https://mlsanity.com/paper/2101.09499/) (2021)
- [Boosting Few-Shot Classification with View-Learnable Contrastive Learning](https://mlsanity.com/paper/2107.09242/) (2021)
- [Multi-Level Contrastive Learning for Few-Shot Problems](https://mlsanity.com/paper/2107.07608/) (2021)

## Built on by (most cited first, 17 of 17)

- [ConDA: Contrastive Domain Adaptation for AI-generated Text Detection](https://mlsanity.com/paper/2309.03992/) (2023)
- [Heterogeneous Contrastive Learning for Foundation Models and Beyond](https://mlsanity.com/paper/2404.00225/) (2024)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification](https://mlsanity.com/paper/2406.17534/) (2024)
- [Effective Structured Prompting by Meta-Learning and Representative Verbalizer](https://mlsanity.com/paper/2306.00618/) (2023)
- [Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Text](https://mlsanity.com/paper/2407.09364/) (2024)
- [Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models](https://mlsanity.com/paper/2406.07001/) (2024)
- [Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification](https://mlsanity.com/paper/2305.14827/) (2023)
- [BYOC: Personalized Few-Shot Classification with Co-Authored Class Descriptions](https://mlsanity.com/paper/2310.06111/) (2023)
- [Across-Game Engagement Modelling via Few-Shot Learning](https://mlsanity.com/paper/2409.13002/) (2024)
- [TART: Improved Few-shot Text Classification Using Task-Adaptive Reference Transformation](https://mlsanity.com/paper/2306.02175/) (2023)
- [Learning Label Hierarchy with Supervised Contrastive Learning](https://mlsanity.com/paper/2402.00232/) (2024)
- [GANs Conditioning Methods: A Survey](https://mlsanity.com/paper/2408.15640/) (2024)
- [DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework](https://mlsanity.com/paper/2312.02532/) (2023)
- [TK-KNN: A Balanced Distance-Based Pseudo Labeling Approach for Semi-Supervised Intent Classification](https://mlsanity.com/paper/2310.11607/) (2023)
- [Visual Analytics for Fine-grained Text Classification Models and Datasets](https://mlsanity.com/paper/2403.15492/) (2024)
- [Improve Meta-learning for Few-Shot Text Classification with All You Can Acquire from the Tasks](https://mlsanity.com/paper/2410.10454/) (2024)

## Abstract (arXiv)

Few-shot text classification has recently been promoted by the meta-learning paradigm which aims to identify target classes with knowledge transferred from source classes with sets of small tasks named episodes. Despite their success, existing works building their meta-learner based on Prototypical Networks are unsatisfactory in learning discriminative text representations between similar classes, which may lead to contradictions during label prediction. In addition, the tasklevel and instance-level overfitting problems in few-shot text classification caused by a few training examples are not sufficiently tackled. In this work, we propose a contrastive learning framework named ContrastNet to tackle both discriminative representation and overfitting problems in few-shot text classification. ContrastNet learns to pull closer text representations belonging to the same class and push away text representations belonging to different classes, while simultaneously introducing unsupervised contrastive regularization at both task-level and instance-level to prevent overfitting. Experiments on 8 few-shot text classification datasets show that ContrastNet outperforms the current state-of-the-art models.