# A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm

arXiv: https://arxiv.org/abs/2310.00708 · HTML: https://mlsanity.com/paper/2310.00708/

## Key facts

- Cited by 9 papers in the map (data through 7 Oct 2026).
- Builds on 50 of its 82 references that are in the map.
- First posted on arXiv 1 Oct 2023.

## Builds on (most cited first, 30 of 50)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [Infinite Mixture Prototypes for Few-Shot Learning](https://mlsanity.com/paper/1902.04552/) (2019)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey](https://mlsanity.com/paper/2007.00753/) (2020)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [Learning with Average Top-k Loss](https://mlsanity.com/paper/1705.08826/) (2017)
- [The limits of min-max optimization algorithms: convergence to spurious non-critical sets](https://mlsanity.com/paper/2006.09065/) (2020)
- [Learning-to-Learn Stochastic Gradient Descent with Biased Regularization](https://mlsanity.com/paper/1903.10399/) (2019)
- [Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis](https://mlsanity.com/paper/2109.14595/) (2021)
- [Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings](https://mlsanity.com/paper/2008.06622/) (2020)

## Built on by (most cited first, 9 of 9)

- [Domain Generalization through Meta-Learning: A Survey](https://mlsanity.com/paper/2404.02785/) (2024)
- [Doubly Mild Generalization for Offline Reinforcement Learning](https://mlsanity.com/paper/2411.07934/) (2024)
- [Model Predictive Task Sampling for Efficient and Robust Adaptation](https://mlsanity.com/paper/2501.11039/) (2025)
- [Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression](https://mlsanity.com/paper/2410.19400/) (2024)
- [Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation](https://mlsanity.com/paper/2407.19523/) (2024)
- [Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments](https://mlsanity.com/paper/2504.19139/) (2025)
- [Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning](https://mlsanity.com/paper/2410.22788/) (2024)
- [Distributionally robust minimization in meta-learning for system identification](https://mlsanity.com/paper/2506.18074/) (2025)
- [PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classification](https://mlsanity.com/paper/2608.15504/) (2026)

## Abstract (arXiv)

Meta learning is a promising paradigm to enable skill transfer across tasks. Most previous methods employ the empirical risk minimization principle in optimization. However, the resulting worst fast adaptation to a subset of tasks can be catastrophic in risk-sensitive scenarios. To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of expected tail risk. We take the two-stage strategy as heuristics to solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.