# Towards Understanding Generalization in Gradient-Based Meta-Learning

arXiv: https://arxiv.org/abs/1907.07287 · HTML: https://mlsanity.com/paper/1907.07287/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2019.

## Built on by (most cited first, 12 of 12)

- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)
- [Meta-Learning with Fewer Tasks through Task Interpolation](https://mlsanity.com/paper/2106.02695/) (2021)
- [Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks](https://mlsanity.com/paper/2102.03832/) (2021)
- [Multimodality in Meta-Learning: A Comprehensive Survey](https://mlsanity.com/paper/2109.13576/) (2021)
- [TAG: Task-based Accumulated Gradients for Lifelong learning](https://mlsanity.com/paper/2105.05155/) (2021)
- [Improving Meta-Learning Generalization with Activation-Based Early-Stopping](https://mlsanity.com/paper/2208.02377/) (2022)
- [TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins](https://mlsanity.com/paper/2606.17660/) (2026)
- [B-SMALL: A Bayesian Neural Network approach to Sparse Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2101.00203/) (2021)
- [Improving Generalization in Meta-Learning via Meta-Gradient Augmentation](https://mlsanity.com/paper/2306.08460/) (2023)
- [Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Method](https://mlsanity.com/paper/2309.14601/) (2023)
- [Accelerating Meta-Learning by Sharing Gradients](https://mlsanity.com/paper/2312.08398/) (2023)
- [Neural Coherence : Find higher performance to out-of-distribution tasks from few samples](https://mlsanity.com/paper/2512.05880/) (2025)

## Abstract (arXiv)

In this work we study generalization of neural networks in gradient-based meta-learning by analyzing various properties of the objective landscapes. We experimentally demonstrate that as meta-training progresses, the meta-test solutions, obtained after adapting the meta-train solution of the model, to new tasks via few steps of gradient-based fine-tuning, become flatter, lower in loss, and further away from the meta-train solution. We also show that those meta-test solutions become flatter even as generalization starts to degrade, thus providing an experimental evidence against the correlation between generalization and flat minima in the paradigm of gradient-based meta-leaning. Furthermore, we provide empirical evidence that generalization to new tasks is correlated with the coherence between their adaptation trajectories in parameter space, measured by the average cosine similarity between task-specific trajectory directions, starting from a same meta-train solution. We also show that coherence of meta-test gradients, measured by the average inner product between the task-specific gradient vectors evaluated at meta-train solution, is also correlated with generalization. Based on these observations, we propose a novel regularizer for MAML and provide experimental evidence for its effectiveness.