# MAML and ANIL Provably Learn Representations

arXiv: https://arxiv.org/abs/2202.03483 · HTML: https://mlsanity.com/paper/2202.03483/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2022.

## Built on by (most cited first, 30 of 30)

- [Transformers as Algorithms: Generalization and Stability in In-context Learning](https://mlsanity.com/paper/2301.07067/) (2023)
- [FedAvg with Fine Tuning: Local Updates Lead to Representation Learning](https://mlsanity.com/paper/2205.13692/) (2022)
- [Learning to Generate Image Embeddings with User-level Differential Privacy](https://mlsanity.com/paper/2211.10844/) (2022)
- [A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem](https://mlsanity.com/paper/2206.08868/) (2022)
- [Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks](https://mlsanity.com/paper/2307.06887/) (2023)
- [Metalearning with Very Few Samples Per Task](https://mlsanity.com/paper/2312.13978/) (2023)
- [MetaNO: How to Transfer Your Knowledge on Learning Hidden Physics](https://mlsanity.com/paper/2301.12095/) (2023)
- [Provable Generalization of Overparameterized Meta-learning Trained with SGD](https://mlsanity.com/paper/2206.09136/) (2022)
- [Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Data](https://mlsanity.com/paper/2311.01420/) (2023)
- [Multitask Learning via Shared Features: Algorithms and Hardness](https://mlsanity.com/paper/2209.03112/) (2022)
- [Convergence of Gradient-based MAML in LQR](https://mlsanity.com/paper/2309.06588/) (2023)
- [Transfer learning for atomistic simulations using GNNs and kernel mean embeddings](https://mlsanity.com/paper/2306.01589/) (2023)
- [Provable unlearning in topic modeling and downstream tasks](https://mlsanity.com/paper/2411.12600/) (2024)
- [ADAGE: A generic two-layer framework for adaptive agent based modelling](https://mlsanity.com/paper/2501.09429/) (2025)
- [Provable Pathways: Learning Multiple Tasks over Multiple Paths](https://mlsanity.com/paper/2303.04338/) (2023)
- [Long-Context Linear System Identification](https://mlsanity.com/paper/2410.05690/) (2024)
- [pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup](https://mlsanity.com/paper/2501.11002/) (2025)
- [Active Representation Learning for General Task Space with Applications in Robotics](https://mlsanity.com/paper/2306.08942/) (2023)
- [Few-shot Multi-Task Learning of Linear Invariant Features with Meta Subspace Pursuit](https://mlsanity.com/paper/2409.02708/) (2024)
- [Provable Meta-Learning with Low-Rank Adaptations](https://mlsanity.com/paper/2410.22264/) (2024)
- [A Statistical Guarantee for Representation Transfer in Multitask Imitation Learning](https://mlsanity.com/paper/2311.01589/) (2023)
- [Cooperative Meta-Learning with Gradient Augmentation](https://mlsanity.com/paper/2406.04639/) (2024)
- [Accelerating Multi-Task Temporal Difference Learning under Low-Rank Representation](https://mlsanity.com/paper/2503.02030/) (2025)
- [Transformers are almost optimal metalearners for linear classification](https://mlsanity.com/paper/2510.19797/) (2025)
- [When More Data Doesn't Help: Limits of Adaptation in Multitask Learning](https://mlsanity.com/paper/2601.20774/) (2026)
- [First-order ANIL provably learns representations despite overparametrization](https://mlsanity.com/paper/2303.01335/) (2023)
- [Plug-and-Play Feature Generation for Few-Shot Medical Image Classification](https://mlsanity.com/paper/2310.09471/) (2023)
- [Dynamics of Meta-learning Representation in the Teacher-student Scenario](https://mlsanity.com/paper/2408.12545/) (2024)
- [Meta-learning of shared linear representations beyond well-specified linear regression](https://mlsanity.com/paper/2501.18975/) (2025)
- [Provable Data Scaling Law for Meta Learning via Complexity Minimization](https://mlsanity.com/paper/2606.02008/) (2026)

## Abstract (arXiv)

Recent empirical evidence has driven conventional wisdom to believe that gradient-based meta-learning (GBML) methods perform well at few-shot learning because they learn an expressive data representation that is shared across tasks. However, the mechanics of GBML have remained largely mysterious from a theoretical perspective. In this paper, we prove that two well-known GBML methods, MAML and ANIL, as well as their first-order approximations, are capable of learning common representation among a set of given tasks. Specifically, in the well-known multi-task linear representation learning setting, they are able to recover the ground-truth representation at an exponentially fast rate. Moreover, our analysis illuminates that the driving force causing MAML and ANIL to recover the underlying representation is that they adapt the final layer of their model, which harnesses the underlying task diversity to improve the representation in all directions of interest. To the best of our knowledge, these are the first results to show that MAML and/or ANIL learn expressive representations and to rigorously explain why they do so.