A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm

arXiv 2310.00708 ↗First posted 1 Oct 2023cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Meta learning is a promising paradigm to enable skill transfer across tasks. Most previous methods employ the empirical risk minimization principle in optimization. However, the resulting worst fast adaptation to a subset of tasks can be catastrophic in risk-sensitive scenarios. To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of expected tail risk. We take the two-stage strategy as heuristics to solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 50 of its 82 references that are in the map. First posted on arXiv 1 Oct 2023.

Builds on

50 of 82 references.

References · 50

2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
2014Explaining and Harnessing Adversarial Examplescited by 7,287
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
2017Prototypical Networks for Few-shot Learningcited by 3,001
2016Matching Networks for One Shot Learningcited by 2,246
2018On First-Order Meta-Learning Algorithmscited by 1,003
2016Adversarial Machine Learning at Scalecited by 934
2019Meta-Learning with Implicit Gradientscited by 523
2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
2017Robust Adversarial Reinforcement Learningcited by 439
2018Conditional Neural Processescited by 421
2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377

38 more

2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
2018Neural Processescited by 307
2018How to train your MAMLcited by 306
2018Probabilistic Model-Agnostic Meta-Learningcited by 254
2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
2018Meta-Learning Probabilistic Inference For Predictioncited by 108
2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
2020Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processescited by 74
2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 71
2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 57
2017Learning with Average Top-k Losscited by 52
2020The limits of min-max optimization algorithms: convergence to spurious non-critical setscited by 50
2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
2022Hypernetworks in Meta-Reinforcement Learningcited by 39
2021Understanding Continual Learning Settings with Data Distribution Drift Analysiscited by 38
2021Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Taskscited by 36
2017Robust Optimization for Non-Convex Objectivescited by 34
2020Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variablescited by 33
2016Minimizing the Maximal Loss: How and Why?cited by 30
2020How Important is the Train-Validation Split in Meta-Learning?cited by 30
2021On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learningcited by 29
2021A Bit More Bayesian: Domain-Invariant Learning with Uncertaintycited by 27
2020Function Contrastive Learning of Transferable Meta-Representationscited by 19
2019Learning Robust Options by Conditional Value at Risk Optimizationcited by 16
2018A Framework for Robot Manipulation: Skill Formalism, Meta Learning and Adaptive Controlcited by 15
2021Variational Multi-Task Learning with Gumbel-Softmax Priorscited by 15
2023Energy-Based Test Sample Adaptation for Domain Generalizationcited by 14
2018Security Analysis and Enhancement of Model Compressed Deep Learning Systems under Adversarial Attackscited by 8
2021Two steps to risk sensitivitycited by 8
2019Adversarially Robust Few-Shot Learning: A Meta-Learning Approachcited by 6
2022Risk-Averse No-Regret Learning in Online Convex Gamescited by 5
2023Large-scale Generative Simulation Artificial Intelligence: the Next Hotspot in Generative AIcited by 5
2021Data-driven deep density estimationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 1.

9 papers.

2024Domain Generalization through Meta-Learning: A Surveycited by 28
2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21
2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 20
2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppressioncited by 17
2024Robust Fast Adaptation from Adversarially Explicit Task Distribution Generationcited by 12
2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 3
2025Distributionally robust minimization in meta-learning for system identificationcited by 2
2026PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classificationcited by 0