# The Effect of Diversity in Meta-Learning

arXiv: https://arxiv.org/abs/2201.11775 · HTML: https://mlsanity.com/paper/2201.11775/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes](https://mlsanity.com/paper/1906.07697/) (2019)
- [Meta-Learning without Memorization](https://mlsanity.com/paper/1912.03820/) (2019)
- [Unsupervised Learning via Meta-Learning](https://mlsanity.com/paper/1810.02334/) (2018)
- [Batch Active Learning Using Determinantal Point Processes](https://mlsanity.com/paper/1906.07975/) (2019)
- [Adaptive Task Sampling for Meta-Learning](https://mlsanity.com/paper/2007.08735/) (2020)
- [Active Mini-Batch Sampling using Repulsive Point Processes](https://mlsanity.com/paper/1804.02772/) (2018)
- [Task Augmentation by Rotating for Meta-Learning](https://mlsanity.com/paper/2003.00804/) (2020)
- [Is Support Set Diversity Necessary for Meta-Learning?](https://mlsanity.com/paper/2011.14048/) (2020)
- [Uncertainty in Multitask Transfer Learning](https://mlsanity.com/paper/1806.07528/) (2018)

## Built on by (most cited first, 14 of 14)

- [Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression](https://mlsanity.com/paper/2306.15063/) (2023)
- [Model Predictive Task Sampling for Efficient and Robust Adaptation](https://mlsanity.com/paper/2501.11039/) (2025)
- [Towards Task Sampler Learning for Meta-Learning](https://mlsanity.com/paper/2307.08924/) (2023)
- [On the Role of Depth and Looping for In-Context Learning with Task Diversity](https://mlsanity.com/paper/2410.21698/) (2024)
- [The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence](https://mlsanity.com/paper/2208.01545/) (2022)
- [Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization](https://mlsanity.com/paper/2306.09222/) (2023)
- [Is Pre-training Truly Better Than Meta-Learning?](https://mlsanity.com/paper/2306.13841/) (2023)
- [DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks](https://mlsanity.com/paper/2503.09679/) (2025)
- [Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling](https://mlsanity.com/paper/2311.02879/) (2023)
- [Meta-Transfer Derm-Diagnosis: Exploring Few-Shot Learning and Transfer Learning for Skin Disease Classification in Long-Tail Distribution](https://mlsanity.com/paper/2404.16814/) (2024)
- [Effects of Structural Allocation of Geometric Task Diversity in Linear Meta-Learning Models](https://mlsanity.com/paper/2509.18349/) (2025)
- [Meta-learning to Address Data Shift in Time Series Classification](https://mlsanity.com/paper/2601.09018/) (2026)
- [PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classification](https://mlsanity.com/paper/2608.15504/) (2026)
- [Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling](https://mlsanity.com/paper/2610.14913/) (2026)

## Abstract (arXiv)

Recent studies show that task distribution plays a vital role in the meta-learner's performance. Conventional wisdom is that task diversity should improve the performance of meta-learning. In this work, we find evidence to the contrary; (i) our experiments draw into question the efficacy of our learned models: similar manifolds can be learned with a subset of the data (lower task diversity). This finding questions the advantage of providing more data to the model, and (ii) adding diversity to the task distribution (higher task diversity) sometimes hinders the model and does not lead to a significant improvement in performance as previously believed. To strengthen our findings, we provide both empirical and theoretical evidence.