# Multi-Level Contrastive Learning for Few-Shot Problems

arXiv: https://arxiv.org/abs/2107.07608 · HTML: https://mlsanity.com/paper/2107.07608/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 33 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 30 of 30)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Cross Attention Network for Few-shot Classification](https://mlsanity.com/paper/1910.07677/) (2019)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Automatically Discovering and Learning New Visual Categories with Ranking Statistics](https://mlsanity.com/paper/2002.05714/) (2020)
- [Finding Task-Relevant Features for Few-Shot Learning by Category Traversal](https://mlsanity.com/paper/1905.11116/) (2019)
- [Learning to Self-Train for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1906.00562/) (2019)
- [Embedding Propagation: Smoother Manifold for Few-Shot Classification](https://mlsanity.com/paper/2003.04151/) (2020)
- [DPGN: Distribution Propagation Graph Network for Few-shot Learning](https://mlsanity.com/paper/2003.14247/) (2020)
- [Diversity with Cooperation: Ensemble Methods for Few-Shot Classification](https://mlsanity.com/paper/1903.11341/) (2019)
- [An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning](https://mlsanity.com/paper/1904.08479/) (2019)
- [TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classification](https://mlsanity.com/paper/2003.06670/) (2020)
- [Semi-Supervised Contrastive Learning with Generalized Contrastive Loss and Its Application to Speaker Recognition](https://mlsanity.com/paper/2006.04326/) (2020)
- [Spatial Contrastive Learning for Few-Shot Classification](https://mlsanity.com/paper/2012.13831/) (2020)
- [Meta-Learned Confidence for Few-shot Learning](https://mlsanity.com/paper/2002.12017/) (2020)
- [Graph-based Interpolation of Feature Vectors for Accurate Few-Shot Classification](https://mlsanity.com/paper/2001.09849/) (2020)
- [Few-Shot Image Classification via Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2008.09942/) (2020)

## Built on by (most cited first, 1 of 1)

- [ContrastNet: A Contrastive Learning Framework for Few-Shot Text Classification](https://mlsanity.com/paper/2305.09269/) (2023)

## Abstract (arXiv)

Contrastive learning is a discriminative approach that aims at grouping similar samples closer and diverse samples far from each other. It it an efficient technique to train an encoder generating distinguishable and informative representations, and it may even increase the encoder's transferability. Most current applications of contrastive learning benefit only a single representation from the last layer of an encoder.In this paper, we propose a multi-level contrasitive learning approach which applies contrastive losses at different layers of an encoder to learn multiple representations from the encoder. Afterward, an ensemble can be constructed to take advantage of the multiple representations for the downstream tasks. We evaluated the proposed method on few-shot learning problems and conducted experiments using the mini-ImageNet and the tiered-ImageNet datasets. Our model achieved the new state-of-the-art results for both datasets, comparing to previous regular, ensemble, and contrastive learing (single-level) based approaches.