# An Information-Geometric Distance on the Space of Tasks

arXiv: https://arxiv.org/abs/2011.00613 · HTML: https://mlsanity.com/paper/2011.00613/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [Probing transfer learning with a model of synthetic correlated datasets](https://mlsanity.com/paper/2106.05418/) (2021)
- [Wasserstein Task Embedding for Measuring Task Similarities](https://mlsanity.com/paper/2208.11726/) (2022)
- [Features are fate: a theory of transfer learning in high-dimensional regression](https://mlsanity.com/paper/2410.08194/) (2024)
- [Lightspeed Geometric Dataset Distance via Sliced Optimal Transport](https://mlsanity.com/paper/2501.18901/) (2025)
- [A picture of the space of typical learnable tasks](https://mlsanity.com/paper/2210.17011/) (2022)
- [Hierarchical Optimal Transport for Comparing Histopathology Datasets](https://mlsanity.com/paper/2204.08324/) (2022)
- [Inducing a hierarchy for multi-class classification problems](https://mlsanity.com/paper/2102.10263/) (2021)
- [Time-Varying Propensity Score to Bridge the Gap between the Past and Present](https://mlsanity.com/paper/2210.01422/) (2022)
- [Meta-Learning for Unsupervised Outlier Detection with Optimal Transport](https://mlsanity.com/paper/2211.00372/) (2022)
- [Generating Synthetic Datasets by Interpolating along Generalized Geodesics](https://mlsanity.com/paper/2306.06866/) (2023)
- [Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics](https://mlsanity.com/paper/2302.02092/) (2023)
- [Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neuroscience](https://mlsanity.com/paper/2308.03175/) (2023)
- [Fisher-Rao distance and pullback SPD cone distances between multivariate normal distributions](https://mlsanity.com/paper/2307.10644/) (2023)
- [Back to the Basics on Predicting Transfer Performance](https://mlsanity.com/paper/2405.20420/) (2024)
- [CLAMS: A System for Zero-Shot Model Selection for Clustering](https://mlsanity.com/paper/2407.11286/) (2024)

## Abstract (arXiv)

This paper prescribes a distance between learning tasks modeled as joint distributions on data and labels. Using tools in information geometry, the distance is defined to be the length of the shortest weight trajectory on a Riemannian manifold as a classifier is fitted on an interpolated task. The interpolated task evolves from the source to the target task using an optimal transport formulation. This distance, which we call the "coupled transfer distance" can be compared across different classifier architectures. We develop an algorithm to compute the distance which iteratively transports the marginal on the data of the source task to that of the target task while updating the weights of the classifier to track this evolving data distribution. We develop theory to show that our distance captures the intuitive idea that a good transfer trajectory is the one that keeps the generalization gap small during transfer, in particular at the end on the target task. We perform thorough empirical validation and analysis across diverse image classification datasets to show that the coupled transfer distance correlates strongly with the difficulty of fine-tuning.