# Classical Sequence Match is a Competitive Few-Shot One-Class Learner

arXiv: https://arxiv.org/abs/2209.06394 · HTML: https://mlsanity.com/paper/2209.06394/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Adversarial Feature Hallucination Networks for Few-Shot Learning](https://mlsanity.com/paper/2003.13193/) (2020)
- [Reasoning about Entailment with Neural Attention](https://mlsanity.com/paper/1509.06664/) (2015)
- [Few-shot Text Classification with Distributional Signatures](https://mlsanity.com/paper/1908.06039/) (2019)
- [Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification](https://mlsanity.com/paper/1906.06678/) (2019)
- [Applying Deep Learning to Answer Selection: A Study and An Open Task](https://mlsanity.com/paper/1508.01585/) (2015)
- [A Compare-Aggregate Model for Matching Text Sequences](https://mlsanity.com/paper/1611.01747/) (2016)
- [Transformer Based Multi-Source Domain Adaptation](https://mlsanity.com/paper/2009.07806/) (2020)
- [Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification](https://mlsanity.com/paper/1811.10999/) (2018)
- [A Compare-Aggregate Model with Latent Clustering for Answer Selection](https://mlsanity.com/paper/1905.12897/) (2019)
- [Few-Shot One-Class Classification via Meta-Learning](https://mlsanity.com/paper/2007.04146/) (2020)
- [Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks](https://mlsanity.com/paper/1910.14326/) (2019)
- [One-Way Prototypical Networks](https://mlsanity.com/paper/1906.00820/) (2019)
- [Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching](https://mlsanity.com/paper/2001.07234/) (2020)
- [Personalized Purchase Prediction of Market Baskets with Wasserstein-Based Sequence Matching](https://mlsanity.com/paper/1905.13131/) (2019)

## Abstract (arXiv)

Nowadays, transformer-based models gradually become the default choice for artificial intelligence pioneers. The models also show superiority even in the few-shot scenarios. In this paper, we revisit the classical methods and propose a new few-shot alternative. Specifically, we investigate the few-shot one-class problem, which actually takes a known sample as a reference to detect whether an unknown instance belongs to the same class. This problem can be studied from the perspective of sequence match. It is shown that with meta-learning, the classical sequence match method, i.e. Compare-Aggregate, significantly outperforms transformer ones. The classical approach requires much less training cost. Furthermore, we perform an empirical comparison between two kinds of sequence match approaches under simple fine-tuning and meta-learning. Meta-learning causes the transformer models' features to have high-correlation dimensions. The reason is closely related to the number of layers and heads of transformer models. Experimental codes and data are available at https://github.com/hmt2014/FewOne