# Charting the Right Manifold: Manifold Mixup for Few-shot Learning

arXiv: https://arxiv.org/abs/1907.12087 · HTML: https://mlsanity.com/paper/1907.12087/

## Key facts

- Cited by 136 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2019.

## Built on by (most cited first, 30 of 136)

- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [SVFormer: Semi-supervised Video Transformer for Action Recognition](https://mlsanity.com/paper/2211.13222/) (2022)
- [Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning](https://mlsanity.com/paper/2102.06605/) (2021)
- [Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification](https://mlsanity.com/paper/2204.04567/) (2022)
- [Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition](https://mlsanity.com/paper/2101.02833/) (2021)
- [A Closer Look at Few-shot Classification Again](https://mlsanity.com/paper/2301.12246/) (2023)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [Rethinking Generalization in Few-Shot Classification](https://mlsanity.com/paper/2206.07267/) (2022)
- [Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?](https://mlsanity.com/paper/2203.17271/) (2022)
- [Transductive Few-shot Learning with Prototype-based Label Propagation by Iterative Graph Refinement](https://mlsanity.com/paper/2304.11598/) (2023)
- [Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning](https://mlsanity.com/paper/2203.09064/) (2022)
- [Transductive Few-Shot Classification on the Oblique Manifold](https://mlsanity.com/paper/2108.04009/) (2021)
- [Supervised Masked Knowledge Distillation for Few-Shot Transformers](https://mlsanity.com/paper/2303.15466/) (2023)
- [LibFewShot: A Comprehensive Library for Few-shot Learning](https://mlsanity.com/paper/2109.04898/) (2021)
- [Task Discrepancy Maximization for Fine-grained Few-Shot Classification](https://mlsanity.com/paper/2207.01376/) (2022)
- [Channel Importance Matters in Few-Shot Image Classification](https://mlsanity.com/paper/2206.08126/) (2022)
- [Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid](https://mlsanity.com/paper/2210.16834/) (2022)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [Reinforced Attention for Few-Shot Learning and Beyond](https://mlsanity.com/paper/2104.04192/) (2021)
- [EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients](https://mlsanity.com/paper/2201.09699/) (2022)
- [Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning](https://mlsanity.com/paper/2101.09499/) (2021)
- [Simple Semantic-Aided Few-Shot Learning](https://mlsanity.com/paper/2311.18649/) (2023)
- [Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning](https://mlsanity.com/paper/2110.09446/) (2021)
- [Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport](https://mlsanity.com/paper/2210.04144/) (2022)
- [The Role of Global Labels in Few-Shot Classification and How to Infer Them](https://mlsanity.com/paper/2108.04055/) (2021)
- [Disentangled Feature Representation for Few-shot Image Classification](https://mlsanity.com/paper/2109.12548/) (2021)
- [Generalized Few-Shot Point Cloud Segmentation Via Geometric Words](https://mlsanity.com/paper/2309.11222/) (2023)

## Abstract (arXiv)

Few-shot learning algorithms aim to learn model parameters capable of adapting to unseen classes with the help of only a few labeled examples. A recent regularization technique - Manifold Mixup focuses on learning a general-purpose representation, robust to small changes in the data distribution. Since the goal of few-shot learning is closely linked to robust representation learning, we study Manifold Mixup in this problem setting. Self-supervised learning is another technique that learns semantically meaningful features, using only the inherent structure of the data. This work investigates the role of learning relevant feature manifold for few-shot tasks using self-supervision and regularization techniques. We observe that regularizing the feature manifold, enriched via self-supervised techniques, with Manifold Mixup significantly improves few-shot learning performance. We show that our proposed method S2M2 beats the current state-of-the-art accuracy on standard few-shot learning datasets like CIFAR-FS, CUB, mini-ImageNet and tiered-ImageNet by 3-8 %. Through extensive experimentation, we show that the features learned using our approach generalize to complex few-shot evaluation tasks, cross-domain scenarios and are robust against slight changes to data distribution.