# On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

arXiv: https://arxiv.org/abs/2102.10454 · HTML: https://mlsanity.com/paper/2102.10454/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 49 references that are in the map.
- First posted on arXiv 20 Feb 2021.

## Builds on (most cited first, 30 of 40)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Provable defenses against adversarial examples via the convex outer adversarial polytope](https://mlsanity.com/paper/1711.00851/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Adversarial Attacks and Defenses in Images, Graphs and Text: A Review](https://mlsanity.com/paper/1909.08072/) (2019)
- [Understanding and Improving Fast Adversarial Training](https://mlsanity.com/paper/2007.02617/) (2020)
- [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://mlsanity.com/paper/1901.08846/) (2019)
- [Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective](https://mlsanity.com/paper/1906.04214/) (2019)
- [You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle](https://mlsanity.com/paper/1905.00877/) (2019)
- [Are Labels Required for Improving Adversarial Robustness?](https://mlsanity.com/paper/1905.13725/) (2019)
- [Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning](https://mlsanity.com/paper/2003.12862/) (2020)
- [When Does Self-supervision Improve Few-shot Learning?](https://mlsanity.com/paper/1910.03560/) (2019)
- [Adversarially robust transfer learning](https://mlsanity.com/paper/1905.08232/) (2019)
- [Structured Adversarial Attack: Towards General Implementation and Better Interpretability](https://mlsanity.com/paper/1808.01664/) (2018)

## Built on by (most cited first, 29 of 29)

- [Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond](https://mlsanity.com/paper/2110.00455/) (2021)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2302.09309/) (2023)
- [Few-Shot Adversarial Prompt Learning on Vision-Language Models](https://mlsanity.com/paper/2403.14774/) (2024)
- [Visual Prompting for Adversarial Robustness](https://mlsanity.com/paper/2210.06284/) (2022)
- [Language-Driven Anchors for Zero-Shot Adversarial Robustness](https://mlsanity.com/paper/2301.13096/) (2023)
- [How and When Adversarial Robustness Transfers in Knowledge Distillation?](https://mlsanity.com/paper/2110.12072/) (2021)
- [Holistic Adversarial Robustness of Deep Learning Models](https://mlsanity.com/paper/2202.07201/) (2022)
- [A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm](https://mlsanity.com/paper/2310.00708/) (2023)
- [Optimizing Robustness and Accuracy in Mixture of Experts: A Dual-Model Approach](https://mlsanity.com/paper/2502.06832/) (2025)
- [Fully Zeroth-Order Bilevel Programming via Gaussian Smoothing](https://mlsanity.com/paper/2404.00158/) (2024)
- [CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic Large Language Models](https://mlsanity.com/paper/2502.20791/) (2025)
- [Long-term Cross Adversarial Training: A Robust Meta-learning Method for Few-shot Classification Tasks](https://mlsanity.com/paper/2106.12900/) (2021)
- [Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning](https://mlsanity.com/paper/2410.22788/) (2024)
- [A Simple Approach to Adversarial Robustness in Few-shot Image Classification](https://mlsanity.com/paper/2204.05432/) (2022)
- [Interpolating Compressed Parameter Subspaces](https://mlsanity.com/paper/2205.09891/) (2022)
- [Adversarial Robustification via Text-to-Image Diffusion Models](https://mlsanity.com/paper/2407.18658/) (2024)
- [Watermarking Graph Neural Networks via Explanations for Ownership Protection](https://mlsanity.com/paper/2501.05614/) (2025)
- [Adversarially Pretrained Transformers May Be Universally Robust In-Context Learners](https://mlsanity.com/paper/2505.14042/) (2025)
- [Robust Generalization with Adaptive Optimal Transport Priors for Decision-Focused Learning](https://mlsanity.com/paper/2602.01427/) (2026)
- [Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2211.15180/) (2022)
- [Deep Adversarial Defense Against Multilevel-Lp Attacks](https://mlsanity.com/paper/2407.09251/) (2024)
- [Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization](https://mlsanity.com/paper/2509.20230/) (2025)
- [MoCo-EA: Exploiting Adversarial Mode Connectivity for Efficient Evolutionary Attacks](https://mlsanity.com/paper/2605.18919/) (2026)
- [Deep Adversarially-Enhanced k-Nearest Neighbors](https://mlsanity.com/paper/2108.06797/) (2021)
- [Learning Transferable Adversarial Robust Representations via Multi-view Consistency](https://mlsanity.com/paper/2210.10485/) (2022)
- [Bridging Models to Defend: A Population-Based Strategy for Robust Adversarial Defense](https://mlsanity.com/paper/2303.10225/) (2023)
- [Contextual Interaction via Primitive-based Adversarial Training For Compositional Zero-shot Learning](https://mlsanity.com/paper/2406.14962/) (2024)
- [Domain-Generalization to Improve Learning in Meta-Learning Algorithms](https://mlsanity.com/paper/2508.09418/) (2025)

## Abstract (arXiv)

Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. It enables us to learn a meta-initialization} of model parameters (that we call meta-model) to rapidly adapt to new tasks using a small amount of labeled training data. Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning. In addition to generalization, robustness is also desired for a meta-model to defend adversarial examples (attacks). Toward promoting adversarial robustness in MAML, we first study WHEN a robustness-promoting regularization should be incorporated, given the fact that MAML adopts a bi-level (fine-tuning vs. meta-update) learning procedure. We show that robustifying the meta-update stage is sufficient to make robustness adapted to the task-specific fine-tuning stage even if the latter uses a standard training protocol. We also make additional justification on the acquired robustness adaptation by peering into the interpretability of neurons' activation maps. Furthermore, we investigate HOW robust regularization can efficiently be designed in MAML. We propose a general but easily-optimized robustness-regularized meta-learning framework, which allows the use of unlabeled data augmentation, fast adversarial attack generation, and computationally-light fine-tuning. In particular, we for the first time show that the auxiliary contrastive learning task can enhance the adversarial robustness of MAML. Finally, extensive experiments are conducted to demonstrate the effectiveness of our proposed methods in robust few-shot learning.