# Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification

arXiv: https://arxiv.org/abs/2209.04702 · HTML: https://mlsanity.com/paper/2209.04702/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 10 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [Adapting Auxiliary Losses Using Gradient Similarity](https://mlsanity.com/paper/1812.02224/) (2018)
- [Self-Supervised Generalisation with Meta Auxiliary Learning](https://mlsanity.com/paper/1901.08933/) (2019)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Diverse Few-Shot Text Classification with Multiple Metrics](https://mlsanity.com/paper/1805.07513/) (2018)
- [Few-shot Text Classification with Distributional Signatures](https://mlsanity.com/paper/1908.06039/) (2019)
- [Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks](https://mlsanity.com/paper/2009.08445/) (2020)
- [Fine-Grained Visual Categorization using Meta-Learning Optimization with Sample Selection of Auxiliary Data](https://mlsanity.com/paper/1807.10916/) (2018)
- [Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification](https://mlsanity.com/paper/2107.12262/) (2021)
- [Self-supervised Regularization for Text Classification](https://mlsanity.com/paper/2103.05231/) (2021)

## Built on by (most cited first, 3 of 3)

- [Mask-guided BERT for Few Shot Text Classification](https://mlsanity.com/paper/2302.10447/) (2023)
- [MetaAdapt: Domain Adaptive Few-Shot Misinformation Detection via Meta Learning](https://mlsanity.com/paper/2305.12692/) (2023)
- [Improve Meta-learning for Few-Shot Text Classification with All You Can Acquire from the Tasks](https://mlsanity.com/paper/2410.10454/) (2024)

## Abstract (arXiv)

Few-shot text classification aims to classify the text under the few-shot scenario. Most of the previous methods adopt optimization-based meta learning to obtain task distribution. However, due to the neglect of matching between the few amount of samples and complicated models, as well as the distinction between useful and useless task features, these methods suffer from the overfitting issue. To address this issue, we propose a novel Adaptive Meta-learner via Gradient Similarity (AMGS) method to improve the model generalization ability to a new task. Specifically, the proposed AMGS alleviates the overfitting based on two aspects: (i) acquiring the potential semantic representation of samples and improving model generalization through the self-supervised auxiliary task in the inner loop, (ii) leveraging the adaptive meta-learner via gradient similarity to add constraints on the gradient obtained by base-learner in the outer loop. Moreover, we make a systematic analysis of the influence of regularization on the entire framework. Experimental results on several benchmarks demonstrate that the proposed AMGS consistently improves few-shot text classification performance compared with the state-of-the-art optimization-based meta-learning approaches.