mlsanityData to 7 Oct 2026

On First-Order Meta-Learning Algorithms

arXiv 1803.02999 ↗First posted 8 Mar 2018cs.LGcited by 1,003

Not on the map. Why?

Abstract (arXiv)

This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.

Key facts. Cited by 1,003 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 1; 2021: 252; 2022: 212; 2023: 174; 2024: 156; 2025: 130; 2026: 76.

1,003 papers.

Sort:

By area:

  1. 2021Robust fine-tuning of zero-shot modelscited by 785
  2. 2021Towards Personalized Federated Learningcited by 462
  3. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  4. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2024Titans: Learning to Memorize at Test Timecited by 303
  7. 2021Gradient Matching for Domain Generalizationcited by 258
  8. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 243
  9. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  10. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  11. 2021Personalized Federated Learning using Hypernetworkscited by 210
  12. 2021Few-shot Image Generation via Cross-domain Correspondencecited by 206
38 more
  1. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  2. 2021Meta-learning via Language Model In-context Tuningcited by 163
  3. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  4. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  5. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  6. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  7. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  8. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  9. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  10. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  11. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  12. 2021Subgraph Federated Learning with Missing Neighbor Generationcited by 132
  13. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  14. 2022ROLAND: Graph Learning Framework for Dynamic Graphscited by 131
  15. 2022DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problemscited by 127
  16. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  17. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  18. 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
  19. 2022Fusing finetuned models for better pretrainingcited by 110
  20. 2021DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detectioncited by 106
  21. 2021Gradient Inversion with Generative Image Priorcited by 101
  22. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  23. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88
  24. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  25. 2023On Penalty-based Bilevel Gradient Descent Methodcited by 87
  26. 2021Multi-Objective Loss Balancing for Physics-Informed Deep Learningcited by 86
  27. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  28. 2021Temporal-Relational CrossTransformers for Few-Shot Action Recognitioncited by 83
  29. 2022Tutorial on amortized optimizationcited by 83
  30. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
  31. 2022COIN++: Neural Compression Across Modalitiescited by 81
  32. 2023Operator Learning with Neural Fields: Tackling PDEs on General Geometriescited by 80
  33. 2022Transformers as Meta-Learners for Implicit Neural Representationscited by 79
  34. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  35. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  36. 2021Generalizable Person Re-identification with Relevance-aware Mixture of Expertscited by 77
  37. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  38. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73

50 most cited of 1,003 All 1,003 (JSON)