mlsanityData to 7 Oct 2026

Generalized Inner Loop Meta-Learning

arXiv 1910.01727 ↗First posted 3 Oct 2019cs.LG stat.MLcited by 92

Not on the map. Why?

Abstract (arXiv)

Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches. Based on this analysis and algorithm, we describe a library of our design, higher, which we share with the community to assist and enable future research into these kinds of meta-learning approaches. We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 29; 2022: 19; 2023: 21; 2024: 14; 2025: 3; 2026: 4.

92 papers.

Sort:

By area:

  1. 2021Fast Model Editing at Scalecited by 491
  2. 2021Learning To Count Everythingcited by 149
  3. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  4. 2022Tutorial on amortized optimizationcited by 83
  5. 2023A Survey of Contextual Optimization Methods for Decision Making under Uncertaintycited by 81
  6. 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 64
  7. 2021A Value-Function-based Interior-point Method for Non-convex Bi-level Optimizationcited by 60
  8. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  9. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  10. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  11. 2021Meta-learning PINN loss functionscited by 51
  12. 2023Modeling rapid language learning by distilling Bayesian priors into artificial neural networkscited by 40
38 more
  1. 2022Betty: An Automatic Differentiation Library for Multilevel Optimizationcited by 27
  2. 2022Gradient-based Bi-level Optimization for Deep Learning: A Surveycited by 26
  3. 2021Value-Function-based Sequential Minimization for Bi-level Optimizationcited by 24
  4. 2022SWL-Adapt: An Unsupervised Domain Adaptation Model with Sample Weight Learning for Cross-User Wearable Human Activity Recognitioncited by 21
  5. 2022Learning to Scaffold: Optimizing Model Explanations for Teachingcited by 20
  6. 2022TorchOpt: An Efficient Library for Differentiable Optimizationcited by 17
  7. 2023Making Scalable Meta Learning Practicalcited by 17
  8. 2021A Meta-Learning Approach for Training Explainable Graph Neural Networkscited by 15
  9. 2021On sensitivity of meta-learning to support datacited by 15
  10. 2022Boosting Natural Language Generation from Instructions with Meta-Learningcited by 15
  11. 2023Exploring Distributional Shifts in Large Language Models for Code Analysiscited by 14
  12. 2021Balanced softmax cross-entropy for incremental learning with and without memorycited by 13
  13. 2025TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendationcited by 13
  14. 2021On the Influence of Masking Policies in Intermediate Pre-trainingcited by 12
  15. 2021Multimodality in Meta-Learning: A Comprehensive Surveycited by 11
  16. 2022Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiationcited by 11
  17. 2023Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?cited by 11
  18. 2022The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalencecited by 10
  19. 2021EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimizationcited by 9
  20. 2022Learning Symbolic Model-Agnostic Loss Functions via Meta-Learningcited by 9
  21. 2022Task Discovery: Finding the Tasks that Neural Networks Generalize oncited by 9
  22. 2023Meta-learning in healthcare: A surveycited by 9
  23. 2021Minimax Optimization with Smooth Algorithmic Adversariescited by 8
  24. 2023Nystrom Method for Accurate and Scalable Implicit Differentiationcited by 8
  25. 2025Meta-Learning Neural Mechanisms rather than Bayesian Priorscited by 8
  26. 2021Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approachcited by 7
  27. 2021Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiationcited by 7
  28. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  29. 2024Metamizer: a versatile neural optimizer for fast and accurate physics simulationscited by 7
  30. 2021Shifting Transformation Learning for Out-of-Distribution Detectioncited by 6
  31. 2022Transfer Dynamics in Emergent Evolutionary Curriculacited by 6
  32. 2023Meta-learning Control Variates: Variance Reduction with Limited Datacited by 6
  33. 2023Is Pre-training Truly Better Than Meta-Learning?cited by 6
  34. 2024Fairness-Aware Meta-Learning via Nash Bargainingcited by 6
  35. 2022OmniPrint: A Configurable Printed Character Synthesizercited by 5
  36. 2023Embarassingly Simple Dataset Distillationcited by 5
  37. 2024Harnessing Meta-Learning for Improving Full-Frame Video Stabilizationcited by 5
  38. 2026Improving Interactive In-Context Learning from Natural Language Feedbackcited by 5

50 most cited of 92 All 92 (JSON)