mlsanityData to 7 Oct 2026

Meta-Learning with a Geometry-Adaptive Preconditioner

arXiv 2304.01552 ↗First posted 4 Apr 2023cs.CV cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Model-agnostic meta-learning (MAML) is one of the most successful meta-learning algorithms. It has a bi-level optimization structure where the outer-loop process learns a shared initialization and the inner-loop process optimizes task-specific weights. Although MAML relies on the standard gradient descent in the inner-loop, recent studies have shown that controlling the inner-loop's gradient descent with a meta-learned preconditioner can be beneficial. Existing preconditioners, however, cannot simultaneously adapt in a task-specific and path-dependent way. Additionally, they do not satisfy the Riemannian metric condition, which can enable the steepest descent learning with preconditioned gradient. In this study, we propose Geometry-Adaptive Preconditioned gradient descent (GAP) that can overcome the limitations in MAML; GAP can efficiently meta-learn a preconditioner that is dependent on task-specific parameters, and its preconditioner can be shown to be a Riemannian metric. Thanks to the two properties, the geometry-adaptive preconditioner is effective for improving the inner-loop optimization. Experiment results show that GAP outperforms the state-of-the-art MAML family and preconditioned gradient descent-MAML (PGD-MAML) family in a variety of few-shot learning tasks. Code is available at: https://github.com/Suhyun777/CVPR23-GAP.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 54 of its 97 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

54 of 97 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2015Trust Region Policy Optimizationcited by 2,503
  9. 2021Convex optimizationcited by 2,479
  10. 2016Matching Networks for One Shot Learningcited by 2,246
  11. 2013Describing Textures in the Wildcited by 2,110
  12. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
38 more
  1. 2019A Closer Look at Few-shot Classificationcited by 679
  2. 2019Meta-Learning with Implicit Gradientscited by 523
  3. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  4. 2018Meta-learning with differentiable closed-form solverscited by 395
  5. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
  6. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  7. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  8. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  9. 2018How to train your MAMLcited by 306
  10. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  11. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  12. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  13. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  14. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
  15. 2019Improved Few-Shot Visual Classificationcited by 114
  16. 2020Incremental Few-Shot Object Detectioncited by 102
  17. 2019Meta-Learning with Warped Gradient Descentcited by 98
  18. 2020Meta-Learning with Adaptive Hyperparameterscited by 79
  19. 2021Cross-domain Few-shot Learning with Task-specific Adapterscited by 79
  20. 2019Meta-Curvaturecited by 70
  21. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  22. 2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 57
  23. 2021Universal Representation Learning from Multiple Domains for Few-shot Classificationcited by 56
  24. 2019Torchmeta: A Meta-Learning library for PyTorchcited by 52
  25. 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
  26. 2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 42
  27. 2019Learning to Forget for Meta-Learningcited by 35
  28. 2021Learning where to learn: Gradient sparsity in meta and continual learningcited by 33
  29. 2021An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systemscited by 32
  30. 2020When Does Preconditioning Help or Hurt Generalization?cited by 30
  31. 2019Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systemscited by 25
  32. 2020Automated Relational Meta-learningcited by 25
  33. 2022HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworkscited by 16
  34. 2021Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGDcited by 14
  35. 2018Training Medical Image Analysis Systems like Radiologistscited by 13
  36. 2020Dithered backprop: A sparse and quantized backpropagation algorithm for more efficient deep neural network trainingcited by 11
  37. 2020Meta-learning the Learning Trends Shared Across Taskscited by 8
  38. 2022On Enforcing Better Conditioned Meta-Learning for Rapid Few-Shot Adaptationcited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 1; 2026: 1.

11 papers.

  1. 2023MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learningcited by 40
  2. 2024Enhancing In-context Learning via Linear Probe Calibrationcited by 16
  3. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  4. 2023Task-Distributionally Robust Data-Free Meta-Learningcited by 4
  5. 2024Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanismscited by 4
  6. 2024Improving Implicit Regularization of SGD with Preconditioning for Least Square Problemscited by 2
  7. 2026Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypescited by 2
  8. 2024Meta-Learning Neural Procedural Biasescited by 1
  9. 2023Learning to Generate Parameters of ConvNets for Unseen Image Datacited by 0
  10. 2024A Simple Task-aware Contrastive Local Descriptor Selection Strategy for Few-shot Learning between inter class and intra classcited by 0
  11. 2025Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learningcited by 0