mlsanityData to 7 Oct 2026

On Data Efficiency of Meta-learning

arXiv 2102.00127 ↗First posted 30 Jan 2021cs.LG cs.AI stat.MLcited by 16

Not on the map. Why?

Abstract (arXiv)

Meta-learning has enabled learning statistical models that can be quickly adapted to new prediction tasks. Motivated by use-cases in personalized federated learning, we study the often overlooked aspect of the modern meta-learning algorithms -- their data efficiency. To shed more light on which methods are more efficient, we use techniques from algorithmic stability to derive bounds on the transfer risk that have important practical implications, indicating how much supervision is needed and how it must be allocated for each method to attain the desired level of generalization. Further, we introduce a new simple framework for evaluating meta-learning methods under a limit on the available supervision, conduct an empirical study of MAML, Reptile, and Protonets, and demonstrate the differences in the behavior of these methods on few-shot and federated learning benchmarks. Finally, we propose active meta-learning, which incorporates active data selection into learning-to-learn, leading to better performance of all methods in the limited supervision regime.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 30 Jan 2021.

Builds on

34 of 52 references.

References · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2019A Closer Look at Few-shot Classificationcited by 679
  11. 2017Federated Multi-Task Learningcited by 669
  12. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
22 more
  1. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  2. 2018Conditional Neural Processescited by 421
  3. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  4. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
  5. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  6. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  7. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  8. 2018Meta-Learning: A Surveycited by 254
  9. 2017One-Shot Imitation Learningcited by 232
  10. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  11. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  12. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  13. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  14. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  15. 2017Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environmentscited by 106
  16. 2019Diverse mini-batch Active Learningcited by 88
  17. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  18. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  19. 2020On the Outsized Importance of Learning Rates in Local Update Methodscited by 34
  20. 2017Learning Algorithms for Active Learningcited by 30
  21. 2017Active One-shot Learningcited by 21
  22. 2017Semi-Supervised and Active Few-Shot Learning with Prototypical Networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 4; 2024: 3; 2025: 2; 2026: 2.

16 papers.

Sort:

  1. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  2. 2021MVDG: A Unified Multi-view Framework for Domain Generalizationcited by 20
  3. 2022Self-Aware Personalized Federated Learningcited by 16
  4. 2024Generalized Coverage for More Robust Low-Budget Active Learningcited by 15
  5. 2021Two Sides of Meta-Learning Evaluation: In vs. Out of Distributioncited by 9
  6. 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
  7. 2023Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resourcescited by 6
  8. 2024Scaling Up Personalized Image Aesthetic Assessment via Task Vector Customizationcited by 6
  9. 2025Balanced Direction from Multifarious Choices: Arithmetic Meta-Learning for Domain Generalizationcited by 5
  10. 2023Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representationscited by 2
  11. 2022Blinder: End-to-end Privacy Protection in Sensing Systems via Personalized Federated Learningcited by 1
  12. 2023Exploring Active Learning in Meta-Learning: Enhancing Context Set Labelingcited by 1
4 more
  1. 2024Decentralized Personalized Federated Learningcited by 1
  2. 2025Detecting Sockpuppetry on Wikipedia Using Meta-Learningcited by 1
  3. 2026Stability beyond Bounded Differences: Sharp Generalization Bounds under Finite Lp Momentscited by 1
  4. 2026Exploring Dualistic Meta-Learning to Enhance Domain Generalization in Open Set Scenarioscited by 0