mlsanityData to 7 Oct 2026

Stateless Neural Meta-Learning using Second-Order Gradients

arXiv 2104.10527 ↗First posted 21 Apr 2021cs.LG cs.AI stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

Deep learning typically requires large data sets and much compute power for each new problem that is learned. Meta-learning can be used to learn a good prior that facilitates quick learning, thereby relaxing these requirements so that new tasks can be learned quicker; two popular approaches are MAML and the meta-learner LSTM. In this work, we compare the two and formally show that the meta-learner LSTM subsumes MAML. Combining this insight with recent empirical findings, we construct a new algorithm (dubbed TURTLE) which is simpler than the meta-learner LSTM yet more expressive than MAML. TURTLE outperforms both techniques at few-shot sine wave regression and image classification on miniImageNet and CUB without any additional hyperparameter tuning, at a computational cost that is comparable with second-order MAML. The key to TURTLE's success lies in the use of second-order gradients, which also significantly increases the performance of the meta-learner LSTM by 1-6% accuracy.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 21 Apr 2021.

Builds on

21 of 39 references.

References · 21

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  4. 2016Matching Networks for One Shot Learningcited by 2,246
  5. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  6. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  7. 2019A Closer Look at Few-shot Classificationcited by 679
  8. 2016Learning to learn by gradient descent by gradient descentcited by 616
  9. 2019Meta-Learning with Implicit Gradientscited by 523
  10. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  11. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  12. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
9 more
  1. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  2. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  3. 2018Meta-Learning: A Surveycited by 254
  4. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  5. 2020A Survey of Deep Meta-Learningcited by 99
  6. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  7. 2020Meta-Learning with Adaptive Hyperparameterscited by 79
  8. 2019Meta-Curvaturecited by 70
  9. 2017Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithmcited by 65

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 0; 2025: 0; 2026: 2.

7 papers.

  1. 2023Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classificationcited by 29
  2. 2022Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classificationcited by 10
  3. 2022Advances in MetaDL: AAAI 2021 challenge and workshopcited by 4
  4. 2023Understanding Transfer Learning and Gradient-Based Meta-Learning Techniquescited by 1
  5. 2023Subspace Adaptation Prior for Few-Shot Learningcited by 1
  6. 2026Layer-wise Derivative Controlled Networkscited by 1
  7. 2026Online Task Adaptation via Self-Organisationcited by 0