mlsanityData to 14 Oct 2026

Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning

arXiv 2206.03996 ↗First posted 8 Jun 2022cs.LG cs.SY eess.SYcited by 58

Not on the map. Why?

Abstract (arXiv)

Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning. Albeit its effectiveness, the optimization of MAML can be challenging due to the innate bilevel problem structure. Specifically, the loss landscape of MAML is much more complex with possibly more saddle points and local minimizers than its empirical risk minimization counterpart. To address this challenge, we leverage the recently invented sharpness-aware minimization and develop a sharpness-aware MAML approach that we term Sharp-MAML. We empirically demonstrate that Sharp-MAML and its computation-efficient variant can outperform the plain-vanilla MAML baseline (e.g., +3% accuracy on Mini-Imagenet). We complement the empirical study with the convergence rate analysis and the generalization bound of Sharp-MAML. To the best of our knowledge, this is the first empirical and theoretical study on sharpness-aware minimization in the context of bilevel learning. The code is available at https://github.com/mominabbass/Sharp-MAML.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 57 of its 75 references that are in the map. First posted on arXiv 8 Jun 2022.

Builds on

57 of 75 references.

References · 57

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  6. 2016Matching Networks for One Shot Learningcited by 2,255
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  8. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  9. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  10. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  11. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  12. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
38 more
  1. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  2. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  3. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  4. 2016Learning to learn by gradient descent by gradient descentcited by 620
  5. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 552
  6. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  7. 2019Meta-Learning with Implicit Gradientscited by 525
  8. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  9. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  10. 2018Meta-learning with differentiable closed-form solverscited by 397
  11. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  12. 2016Learning to reinforcement learncited by 377
  13. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  14. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  15. 2018How to train your MAMLcited by 309
  16. 2017Meta Networkscited by 269
  17. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 260
  18. 2018Meta-Learning: A Surveycited by 256
  19. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  20. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
  21. 2016Learning to Optimizecited by 195
  22. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  23. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  24. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  25. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  26. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  27. 2016Towards a Neural Statisticiancited by 121
  28. 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
  29. 2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
  30. 2019Meta-Learning without Memorizationcited by 92
  31. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  32. 2018Unsupervised Learning via Meta-Learningcited by 82
  33. 2019Meta-Curvaturecited by 70
  34. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  35. 2017Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithmcited by 65
  36. 2017Learning to Remember Rare Eventscited by 63
  37. 2020Tracking by Instance Detection: A Meta-Learning Approachcited by 61
  38. 2019Torchmeta: A Meta-Learning library for PyTorchcited by 52

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 24; 2024: 18; 2025: 12; 2026: 2.

58 papers.

Sort:

By area:

  1. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  2. 2023An Adaptive Policy to Employ Sharpness-Aware Minimizationcited by 43
  3. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  4. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  5. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  6. 2022Escaping Saddle Points for Effective Generalization on Class-Imbalanced Datacited by 28
  7. 2023Normalization Layers Are All That Sharpness-Aware Minimization Needscited by 27
  8. 2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 21
  9. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
  10. 2024Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQRcited by 17
  11. 2023Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Termcited by 16
  12. 2024Enhancing In-context Learning via Linear Probe Calibrationcited by 16
38 more
  1. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  2. 2022Beyond Losses Reweighting: Empowering Multi-Task Learning via the Generalization Perspectivecited by 15
  3. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
  4. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 12
  5. 2023Meta-Learning with a Geometry-Adaptive Preconditionercited by 11
  6. 2023Towards Task Sampler Learning for Meta-Learningcited by 11
  7. 2023Flat Seeking Bayesian Neural Networkscited by 10
  8. 2023A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigmcited by 10
  9. 2024Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problemscited by 10
  10. 2024Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learningcited by 9
  11. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  12. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
  13. 2023Optimal Transport Model Distributional Robustnesscited by 5
  14. 2023Convergence of Gradient-based MAML in LQRcited by 5
  15. 2024MetaAug: Meta-Data Augmentation for Post-Training Quantizationcited by 5
  16. 2024Improving Generalization with Flat Hilbert Bayesian Inferencecited by 5
  17. 2024(FL)2: Overcoming Few Labels in Federated Semi-Supervised Learningcited by 5
  18. 2023Scalable Bayesian Meta-Learning through Generalized Implicit Gradientscited by 4
  19. 2024Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classificationcited by 4
  20. 2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 4
  21. 2025Curvature Learning for Generalization of Hyperbolic Neural Networkscited by 4
  22. 2025Sharpness-Guided Group Relative Policy Optimization via Probability Shapingcited by 4
  23. 2024Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Searchcited by 3
  24. 2025Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Modelscited by 3
  25. 2023Towards Label-Efficient Incremental Learning: A Surveycited by 2
  26. 2023EMO: Episodic Memory Optimization for Few-Shot Meta-Learningcited by 2
  27. 2024Agnostic Sharpness-Aware Minimizationcited by 2
  28. 2023Sharpness & Shift-Aware Self-Supervised Learningcited by 1
  29. 2024Diversity-Aware Agnostic Ensemble of Sharpness Minimizerscited by 1
  30. 2024Cooperative Meta-Learning with Gradient Augmentationcited by 1
  31. 2024Rethinking Meta-Learning from a Learning Lenscited by 1
  32. 2025Towards Better Generalization via Distributional Input Projection Networkcited by 1
  33. 2025SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbationcited by 1
  34. 2026MAML-KT: Addressing Cold Start Problem in Knowledge Tracing for New Students via Few-Shot Model-Agnostic Meta Learningcited by 1
  35. 2023G-Mix: A Generalized Mixup Learning Framework Towards Flat Minimacited by 0
  36. 2023Sharpness-Aware Teleportation on Riemannian Manifoldscited by 0
  37. 2023Generalization Bounds for Robust Contrastive Learning: From Theory to Practicecited by 0
  38. 2024Helen: Optimizing CTR Prediction Models with Frequency-wise Hessian Eigenvalue Regularizationcited by 0

50 most cited of 58 All 58 (JSON)