# Theoretical bounds on estimation error for meta-learning

arXiv: https://arxiv.org/abs/2010.07140 · HTML: https://mlsanity.com/paper/2010.07140/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [Advances and Challenges in Meta-Learning: A Technical Review](https://mlsanity.com/paper/2307.04722/) (2023)
- [How Fine-Tuning Allows for Effective Meta-Learning](https://mlsanity.com/paper/2105.02221/) (2021)
- [Towards Sample-efficient Overparameterized Meta-learning](https://mlsanity.com/paper/2201.06142/) (2022)
- [A Closer Look at Prototype Classifier for Few-shot Image Classification](https://mlsanity.com/paper/2110.05076/) (2021)
- [Metalearning with Very Few Samples Per Task](https://mlsanity.com/paper/2312.13978/) (2023)
- [An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning](https://mlsanity.com/paper/2101.08390/) (2021)
- [Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimation](https://mlsanity.com/paper/2403.19720/) (2024)
- [ReFine: Boosting Time Series Prediction of Extreme Events by Reweighting and Fine-tuning](https://mlsanity.com/paper/2409.14232/) (2024)
- [On the ERM Principle in Meta-Learning](https://mlsanity.com/paper/2411.17898/) (2024)

## Abstract (arXiv)

Machine learning models have traditionally been developed under the assumption that the training and test distributions match exactly. However, recent success in few-shot learning and related problems are encouraging signs that these models can be adapted to more realistic settings where train and test distributions differ. Unfortunately, there is severely limited theoretical support for these algorithms and little is known about the difficulty of these problems. In this work, we provide novel information-theoretic lower-bounds on minimax rates of convergence for algorithms that are trained on data from multiple sources and tested on novel data. Our bounds depend intuitively on the information shared between sources of data, and characterize the difficulty of learning in this setting for arbitrary algorithms. We demonstrate these bounds on a hierarchical Bayesian model of meta-learning, computing both upper and lower bounds on parameter estimation via maximum-a-posteriori inference.