mlsanityData to 7 Oct 2026

Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD

arXiv 2109.07497 ↗First posted 15 Sep 2021cs.LG cs.AI cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

We propose a new computationally-efficient first-order algorithm for Model-Agnostic Meta-Learning (MAML). The key enabling technique is to interpret MAML as a bilevel optimization (BLO) problem and leverage the sign-based SGD(signSGD) as a lower-level optimizer of BLO. We show that MAML, through the lens of signSGD-oriented BLO, naturally yields an alternating optimization scheme that just requires first-order gradients of a learned meta-model. We term the resulting MAML algorithm Sign-MAML. Compared to the conventional first-order MAML (FO-MAML) algorithm, Sign-MAML is theoretically-grounded as it does not impose any assumption on the absence of second-order derivatives during meta training. In practice, we show that Sign-MAML outperforms FO-MAML in various few-shot image classification tasks, and compared to MAML, it achieves a much more graceful tradeoff between classification accuracy and computation efficiency.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

28 of 36 references.

References · 28

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2016Matching Networks for One Shot Learningcited by 2,246
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  10. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  11. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  12. 2019Meta-Learning with Implicit Gradientscited by 523
16 more
  1. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  2. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  3. 2018How to train your MAMLcited by 306
  4. 2018Meta-Learning: A Surveycited by 254
  5. 2018Approximation Methods for Bilevel Programmingcited by 231
  6. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  7. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 160
  8. 2020On the Iteration Complexity of Hypergradient Computationcited by 154
  9. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  10. 2016On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimizationcited by 108
  11. 2019Meta-Learning of Neural Architectures for Few-Shot Learningcited by 73
  12. 2020learn2learn: A Library for Meta-Learning Researchcited by 69
  13. 2020Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameterscited by 65
  14. 2019ES-MAML: Simple Hessian-Free Meta Learningcited by 57
  15. 2019Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Taskscited by 48
  16. 2020When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applicationscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 4; 2024: 2; 2025: 2.

14 papers.

Sort:

  1. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  2. 2022HyperShot: Few-Shot Learning by Kernel HyperNetworkscited by 24
  3. 2022HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworkscited by 16
  4. 2022Maximizing Communication Efficiency for Large-scale Training via 0/1 Adamcited by 15
  5. 2022Adaptive Fine-Grained Sketch-Based Image Retrievalcited by 15
  6. 2023Meta-Learning with a Geometry-Adaptive Preconditionercited by 11
  7. 2023Meta-learning in healthcare: A surveycited by 9
  8. 2024MetaAug: Meta-Data Augmentation for Post-Training Quantizationcited by 5
  9. 2023BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimizationcited by 4
  10. 2024BiSSL: Enhancing the Alignment Between Self-Supervised Pretraining and Downstream Fine-Tuning via Bilevel Optimizationcited by 3
  11. 2022FedBC: Calibrating Global and Local Models via Federated Learning Beyond Consensuscited by 2
  12. 2025Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgmentcited by 1
2 more
  1. 2022Hypernetwork approach to Bayesian MAMLcited by 0
  2. 2025Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraintscited by 0