# Generating Synthetic Datasets by Interpolating along Generalized Geodesics

arXiv: https://arxiv.org/abs/2306.06866 · HTML: https://mlsanity.com/paper/2306.06866/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark](https://mlsanity.com/paper/1910.04867/) (2019)
- [Optimal transport mapping via input convex neural networks](https://mlsanity.com/paper/1908.10962/) (2019)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [Geometric Dataset Distances via Optimal Transport](https://mlsanity.com/paper/2002.02923/) (2020)
- [Neural Optimal Transport](https://mlsanity.com/paper/2201.12220/) (2022)
- [Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm](https://mlsanity.com/paper/1802.04367/) (2018)
- [Fair Mixup: Fairness via Interpolation](https://mlsanity.com/paper/2103.06503/) (2021)
- [GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks](https://mlsanity.com/paper/1810.10863/) (2018)
- [Proximal Optimal Transport Modeling of Population Dynamics](https://mlsanity.com/paper/2106.06345/) (2021)
- [Entropic estimation of optimal transport maps](https://mlsanity.com/paper/2109.12004/) (2021)
- [Generative Modeling with Optimal Transport Maps](https://mlsanity.com/paper/2110.02999/) (2021)
- [Large-Scale Wasserstein Gradient Flows](https://mlsanity.com/paper/2106.00736/) (2021)
- [Supervised Training of Conditional Monge Maps](https://mlsanity.com/paper/2206.14262/) (2022)
- [Variational Wasserstein gradient flow](https://mlsanity.com/paper/2112.02424/) (2021)
- [Continuous Wasserstein-2 Barycenter Estimation without Minimax Optimization](https://mlsanity.com/paper/2102.01752/) (2021)
- [Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks](https://mlsanity.com/paper/2007.04462/) (2020)
- [Meta-Learning with Fewer Tasks through Task Interpolation](https://mlsanity.com/paper/2106.02695/) (2021)
- [Scalable Bayes via Barycenter in Wasserstein Space](https://mlsanity.com/paper/1508.05880/) (2015)
- [SpinalNet: Deep Neural Network with Gradual Input](https://mlsanity.com/paper/2007.03347/) (2020)
- [Neural Monge Map estimation and its applications](https://mlsanity.com/paper/2106.03812/) (2021)
- [Neural Optimal Transport with General Cost Functionals](https://mlsanity.com/paper/2205.15403/) (2022)
- [A Data-Based Perspective on Transfer Learning](https://mlsanity.com/paper/2207.05739/) (2022)
- [Wasserstein Iterative Networks for Barycenter Estimation](https://mlsanity.com/paper/2201.12245/) (2022)
- [Dataset Dynamics via Gradient Flows in Probability Space](https://mlsanity.com/paper/2010.12760/) (2020)
- [An Information-Geometric Distance on the Space of Tasks](https://mlsanity.com/paper/2011.00613/) (2020)

## Built on by (most cited first, 5 of 5)

- [Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equation](https://mlsanity.com/paper/2402.18839/) (2024)
- [Analyzing Quality, Bias, and Performance in Text-to-Image Generative Models](https://mlsanity.com/paper/2407.00138/) (2024)
- [Wasserstein Geodesic Generator for Conditional Distributions](https://mlsanity.com/paper/2308.10145/) (2023)
- [A primer on optimal transport for causal inference with observational data](https://mlsanity.com/paper/2503.07811/) (2025)
- [CTRL Your Shift: Clustered Transfer Residual Learning for Many Small Datasets](https://mlsanity.com/paper/2508.11144/) (2025)

## Abstract (arXiv)

Data for pretraining machine learning models often consists of collections of heterogeneous datasets. Although training on their union is reasonable in agnostic settings, it might be suboptimal when the target domain -- where the model will ultimately be used -- is known in advance. In that case, one would ideally pretrain only on the dataset(s) most similar to the target one. Instead of limiting this choice to those datasets already present in the pretraining collection, here we explore extending this search to all datasets that can be synthesized as `combinations' of them. We define such combinations as multi-dataset interpolations, formalized through the notion of generalized geodesics from optimal transport (OT) theory. We compute these geodesics using a recent notion of distance between labeled datasets, and derive alternative interpolation schemes based on it: using either barycentric projections or optimal transport maps, the latter computed using recent neural OT methods. These methods are scalable, efficient, and -- notably -- can be used to interpolate even between datasets with distinct and unrelated label sets. Through various experiments in transfer learning in computer vision, we demonstrate this is a promising new approach for targeted on-demand dataset synthesis.