# Meta-Learning Requires Meta-Augmentation

arXiv: https://arxiv.org/abs/2007.05549 · HTML: https://mlsanity.com/paper/2007.05549/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 30 of 57)

- [Symbol tuning improves in-context learning in language models](https://mlsanity.com/paper/2305.08298/) (2023)
- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)
- [Meta-Learning with Fewer Tasks through Task Interpolation](https://mlsanity.com/paper/2106.02695/) (2021)
- [Same State, Different Task: Continual Reinforcement Learning without Interference](https://mlsanity.com/paper/2106.02940/) (2021)
- [Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment](https://mlsanity.com/paper/2104.05632/) (2021)
- [Fixing Model Bugs with Natural Language Patches](https://mlsanity.com/paper/2211.03318/) (2022)
- [Meta-learning with an Adaptive Task Scheduler](https://mlsanity.com/paper/2110.14057/) (2021)
- [Meta Learning for Natural Language Processing: A Survey](https://mlsanity.com/paper/2205.01500/) (2022)
- [How to Train Your MAML to Excel in Few-Shot Classification](https://mlsanity.com/paper/2106.16245/) (2021)
- [Position: Leverage Foundational Models for Black-Box Optimization](https://mlsanity.com/paper/2405.03547/) (2024)
- [Few-Shot Learning with Class Imbalance](https://mlsanity.com/paper/2101.02523/) (2021)
- [What Matters For Meta-Learning Vision Regression Tasks?](https://mlsanity.com/paper/2203.04905/) (2022)
- [Learning Deep Time-index Models for Time Series Forecasting](https://mlsanity.com/paper/2207.06046/) (2022)
- [Neural Processes with Stochastic Attention: Paying more attention to the context dataset](https://mlsanity.com/paper/2204.05449/) (2022)
- [Model-Based Offline Meta-Reinforcement Learning with Regularization](https://mlsanity.com/paper/2202.02929/) (2022)
- [Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation](https://mlsanity.com/paper/2407.19523/) (2024)
- [Evolution and The Knightian Blindspot of Machine Learning](https://mlsanity.com/paper/2501.13075/) (2025)
- [MAML is a Noisy Contrastive Learner in Classification](https://mlsanity.com/paper/2106.15367/) (2021)
- [Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation](https://mlsanity.com/paper/2203.11670/) (2022)
- [Hacking Task Confounder in Meta-Learning](https://mlsanity.com/paper/2312.05771/) (2023)
- [Multimodal Meta-Learning for Time Series Regression](https://mlsanity.com/paper/2108.02842/) (2021)
- [Set-based Meta-Interpolation for Few-Task Meta-Learning](https://mlsanity.com/paper/2205.09990/) (2022)
- [AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentation](https://mlsanity.com/paper/2405.11467/) (2024)
- [Meta-GPS++: Enhancing Graph Meta-Learning with Contrastive Learning and Self-Training](https://mlsanity.com/paper/2407.14732/) (2024)
- [Diverse Preference Augmentation with Multiple Domains for Cold-start Recommendations](https://mlsanity.com/paper/2204.00327/) (2022)
- [Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile](https://mlsanity.com/paper/2206.01944/) (2022)
- [Instance Generation for Meta-Black-Box Optimization through Latent Space Reverse Engineering](https://mlsanity.com/paper/2509.15810/) (2025)
- [A Meta-GNN approach to personalized seizure detection and classification](https://mlsanity.com/paper/2211.02642/) (2022)
- [Dual-level Mixup for Graph Few-shot Learning with Fewer Tasks](https://mlsanity.com/paper/2502.14158/) (2025)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)

## Abstract (arXiv)

Meta-learning algorithms aim to learn two components: a model that predicts targets for a task, and a base learner that quickly updates that model when given examples from a new task. This additional level of learning can be powerful, but it also creates another potential source for overfitting, since we can now overfit in either the model or the base learner. We describe both of these forms of metalearning overfitting, and demonstrate that they appear experimentally in common meta-learning benchmarks. We then use an information-theoretic framework to discuss meta-augmentation, a way to add randomness that discourages the base learner and model from learning trivial solutions that do not generalize to new tasks. We demonstrate that meta-augmentation produces large complementary benefits to recently proposed meta-regularization techniques.