# Is Support Set Diversity Necessary for Meta-Learning?

arXiv: https://arxiv.org/abs/2011.14048 · HTML: https://mlsanity.com/paper/2011.14048/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2020.

## Built on by (most cited first, 14 of 14)

- [On sensitivity of meta-learning to support data](https://mlsanity.com/paper/2110.13953/) (2021)
- [The Effect of Diversity in Meta-Learning](https://mlsanity.com/paper/2201.11775/) (2022)
- [A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomness](https://mlsanity.com/paper/2312.01082/) (2023)
- [ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classification](https://mlsanity.com/paper/2112.11916/) (2021)
- [Two Sides of Meta-Learning Evaluation: In vs. Out of Distribution](https://mlsanity.com/paper/2102.11503/) (2021)
- [MetaMedSeg: Volumetric Meta-learning for Few-Shot Organ Segmentation](https://mlsanity.com/paper/2109.09734/) (2021)
- [Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks](https://mlsanity.com/paper/2410.13577/) (2024)
- [Invariant Meta Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2301.11779/) (2023)
- [Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling](https://mlsanity.com/paper/2311.02879/) (2023)
- [On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices](https://mlsanity.com/paper/2402.12817/) (2024)
- [Robust Meta Learning for Image based tasks](https://mlsanity.com/paper/2301.12698/) (2023)
- [Unsupervised Representation Learning to Aid Semi-Supervised Meta Learning](https://mlsanity.com/paper/2310.13085/) (2023)
- [Meta-Task: A Method-Agnostic Framework for Learning to Regularize in Few-Shot Learning](https://mlsanity.com/paper/2402.18599/) (2024)
- [Meta-learning to Address Data Shift in Time Series Classification](https://mlsanity.com/paper/2601.09018/) (2026)

## Abstract (arXiv)

Meta-learning is a popular framework for learning with limited data in which an algorithm is produced by training over multiple few-shot learning tasks. For classification problems, these tasks are typically constructed by sampling a small number of support and query examples from a subset of the classes. While conventional wisdom is that task diversity should improve the performance of meta-learning, in this work we find evidence to the contrary: we propose a modification to traditional meta-learning approaches in which we keep the support sets fixed across tasks, thus reducing task diversity. Surprisingly, we find that not only does this modification not result in adverse effects, it almost always improves the performance for a variety of datasets and meta-learning methods. We also provide several initial analyses to understand this phenomenon. Our work serves to: (i) more closely investigate the effect of support set construction for the problem of meta-learning, and (ii) suggest a simple, general, and competitive baseline for few-shot learning.