mlsanityData to 7 Oct 2026

Meta-Learning with Adaptive Hyperparameters

arXiv 2011.00209 ↗First posted 31 Oct 2020cs.LG cs.CVcited by 79

Not on the map. Why?

Abstract (arXiv)

Despite its popularity, several recent works question the effectiveness of MAML when test tasks are different from training tasks, thus suggesting various task-conditioned methodology to improve the initialization. Instead of searching for better task-aware initialization, we focus on a complementary factor in MAML framework, inner-loop optimization (or fast adaptation). Consequently, we propose a new weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can adaptively generate per-step hyperparameters: learning rate and weight decay coefficients. The experimental results validate that the Adaptive Learning of hyperparameters for Fast Adaptation (ALFA) is the equally important ingredient that was often neglected in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 13; 2023: 23; 2024: 17; 2025: 9; 2026: 4.

79 papers.

Sort:

By area:

  1. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
  2. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  3. 2021Rectifying the Shortcut Learning of Background for Few-Shot Learningcited by 51
  4. 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
  5. 2023A Closer Look at Few-shot Classification Againcited by 46
  6. 2023MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learningcited by 40
  7. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  8. 2024Diffusion-Based Neural Network Weights Generationcited by 38
  9. 2024B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasonerscited by 26
  10. 2022Meta Knowledge Distillationcited by 20
  11. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 17
  12. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
38 more
  1. 2022Data-Efficient Brain Connectome Analysis via Multi-Task Meta-Learningcited by 15
  2. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  3. 2023Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentationcited by 12
  4. 2021MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learningcited by 11
  5. 2023Meta-Learning with a Geometry-Adaptive Preconditionercited by 11
  6. 2023LFS-GAN: Lifelong Few-Shot Image Generationcited by 11
  7. 2024Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learningcited by 11
  8. 2022Multidimensional Belief Quantification for Label-Efficient Meta-Learningcited by 9
  9. 2022Learning Cross-Image Object Semantic Relation in Transformer for Few-Shot Fine-Grained Image Classificationcited by 9
  10. 2023Dataset Distillation via Adversarial Prediction Matchingcited by 9
  11. 2021Revisiting Local Descriptor for Improved Few-Shot Classificationcited by 8
  12. 2021Stateless Neural Meta-Learning using Second-Order Gradientscited by 7
  13. 2021Prototype Completion for Few-Shot Learningcited by 7
  14. 2022Stochastic Deep Networks with Linear Competing Units for Model-Agnostic Meta-Learningcited by 7
  15. 2021MetaCloth: Learning Unseen Tasks of Dense Fashion Landmark Detection from a Few Samplescited by 6
  16. 2021ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learningcited by 4
  17. 2021SGMNet: Scene Graph Matching Network for Few-Shot Remote Sensing Scene Classificationcited by 4
  18. 2021Contextual Gradient Scaling for Few-Shot Learningcited by 4
  19. 2023Meta-Learning Adaptive Loss Functionscited by 4
  20. 2023LSFSL: Leveraging Shape Information in Few-shot Learningcited by 4
  21. 2023Meta-Learning Strategies through Value Maximization in Neural Networkscited by 4
  22. 2024MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependencecited by 4
  23. 2023Improve Noise Tolerance of Robust Loss via Noise-Awarenesscited by 3
  24. 2023AdaSelection: Accelerating Deep Learning Training through Data Subsamplingcited by 3
  25. 2024Mind the Gap Between Prototypes and Images in Cross-domain Finetuningcited by 3
  26. 2025Curvature Learning for Generalization of Hyperbolic Neural Networkscited by 3
  27. 2025LLMs as In-Context Meta-Learners for Model and Hyperparameter Selectioncited by 3
  28. 2021Rapid Model Architecture Adaption for Meta-Learningcited by 2
  29. 2022MetaDT: Meta Decision Tree with Class Hierarchy for Interpretable Few-Shot Learningcited by 2
  30. 2023Task Weighting in Meta-learning with Trajectory Optimisationcited by 2
  31. 2023Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts using Meta-Learning Hypernetworkscited by 2
  32. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  33. 2024Meta-Learning with Heterogeneous Taskscited by 2
  34. 2025Ready2Unlearn: A Learning-Time Approach for Preparing Models with Future Unlearning Readinesscited by 2
  35. 2025Hyperbolic Dual Feature Augmentation for Open-Environmentcited by 2
  36. 2025Few-Shot Learning in Video and 3D Object Detection: A Surveycited by 2
  37. 2026Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shiftcited by 2
  38. 2022Interval Bound Interpolation for Few-shot Learning with Few Taskscited by 1

50 most cited of 79 All 79 (JSON)