# On the Subspace Structure of Gradient-Based Meta-Learning

arXiv: https://arxiv.org/abs/2207.03804 · HTML: https://mlsanity.com/paper/2207.03804/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 14 of 14)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Meta-Learning with Warped Gradient Descent](https://mlsanity.com/paper/1909.00025/) (2019)
- [Learning Neural Network Subspaces](https://mlsanity.com/paper/2102.10472/) (2021)
- [Meta-Curvature](https://mlsanity.com/paper/1902.03356/) (2019)
- [Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks](https://mlsanity.com/paper/2002.06753/) (2020)
- [Revisiting "Qualitatively Characterizing Neural Network Optimization Problems"](https://mlsanity.com/paper/2012.06898/) (2020)
- [Learning a subspace of policies for online adaptation in Reinforcement Learning](https://mlsanity.com/paper/2110.05169/) (2021)
- [GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability](https://mlsanity.com/paper/2011.07407/) (2020)

## Abstract (arXiv)

In this work we provide an analysis of the distribution of the post-adaptation parameters of Gradient-Based Meta-Learning (GBML) methods. Previous work has noticed how, for the case of image-classification, this adaptation only takes place on the last layers of the network. We propose the more general notion that parameters are updated over a low-dimensional subspace of the same dimensionality as the task-space and show that this holds for regression as well. Furthermore, the induced subspace structure provides a method to estimate the intrinsic dimension of the space of tasks of common few-shot learning datasets.