mlsanityData to 7 Oct 2026

MAML and ANIL Provably Learn Representations

arXiv 2202.03483 ↗First posted 7 Feb 2022cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Recent empirical evidence has driven conventional wisdom to believe that gradient-based meta-learning (GBML) methods perform well at few-shot learning because they learn an expressive data representation that is shared across tasks. However, the mechanics of GBML have remained largely mysterious from a theoretical perspective. In this paper, we prove that two well-known GBML methods, MAML and ANIL, as well as their first-order approximations, are capable of learning common representation among a set of given tasks. Specifically, in the well-known multi-task linear representation learning setting, they are able to recover the ground-truth representation at an exponentially fast rate. Moreover, our analysis illuminates that the driving force causing MAML and ANIL to recover the underlying representation is that they adapt the final layer of their model, which harnesses the underlying task diversity to improve the representation in all directions of interest. To the best of our knowledge, these are the first results to show that MAML and/or ANIL learn expressive representations and to rigorously explain why they do so.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2022.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 12; 2024: 6; 2025: 5; 2026: 2.

30 papers.

Sort:

By area:

  1. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 220
  2. 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
  3. 2022Learning to Generate Image Embeddings with User-level Differential Privacycited by 23
  4. 2022A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problemcited by 17
  5. 2023Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networkscited by 15
  6. 2023Metalearning with Very Few Samples Per Taskcited by 11
  7. 2023MetaNO: How to Transfer Your Knowledge on Learning Hidden Physicscited by 10
  8. 2022Provable Generalization of Overparameterized Meta-learning Trained with SGDcited by 9
  9. 2023Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Datacited by 7
  10. 2022Multitask Learning via Shared Features: Algorithms and Hardnesscited by 5
  11. 2023Convergence of Gradient-based MAML in LQRcited by 5
  12. 2023Transfer learning for atomistic simulations using GNNs and kernel mean embeddingscited by 4
18 more
  1. 2024Provable unlearning in topic modeling and downstream taskscited by 4
  2. 2025ADAGE: A generic two-layer framework for adaptive agent based modellingcited by 4
  3. 2023Provable Pathways: Learning Multiple Tasks over Multiple Pathscited by 3
  4. 2024Long-Context Linear System Identificationcited by 3
  5. 2023Active Representation Learning for General Task Space with Applications in Roboticscited by 2
  6. 2024Few-shot Multi-Task Learning of Linear Invariant Features with Meta Subspace Pursuitcited by 2
  7. 2024Provable Meta-Learning with Low-Rank Adaptationscited by 2
  8. 2025pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixupcited by 2
  9. 2023A Statistical Guarantee for Representation Transfer in Multitask Imitation Learningcited by 1
  10. 2024Cooperative Meta-Learning with Gradient Augmentationcited by 1
  11. 2025Accelerating Multi-Task Temporal Difference Learning under Low-Rank Representationcited by 1
  12. 2025Transformers are almost optimal metalearners for linear classificationcited by 1
  13. 2026When More Data Doesn't Help: Limits of Adaptation in Multitask Learningcited by 1
  14. 2023First-order ANIL provably learns representations despite overparametrizationcited by 0
  15. 2023Plug-and-Play Feature Generation for Few-Shot Medical Image Classificationcited by 0
  16. 2024Dynamics of Meta-learning Representation in the Teacher-student Scenariocited by 0
  17. 2025Meta-learning of shared linear representations beyond well-specified linear regressioncited by 0
  18. 2026Provable Data Scaling Law for Meta Learning via Complexity Minimizationcited by 0