# Unbalanced minibatch Optimal Transport; applications to Domain Adaptation

arXiv: https://arxiv.org/abs/2103.03606 · HTML: https://mlsanity.com/paper/2103.03606/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 64 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 30 of 35)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1803.10081/) (2018)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [Partial Adversarial Domain Adaptation](https://mlsanity.com/paper/1808.04205/) (2018)
- [Stochastic subgradient method converges on tame functions](https://mlsanity.com/paper/1804.07795/) (2018)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Missing Data Imputation using Optimal Transport](https://mlsanity.com/paper/2002.03860/) (2020)
- [Subspace Robust Wasserstein Distances](https://mlsanity.com/paper/1901.08949/) (2019)
- [Learning to Transfer Examples for Partial Domain Adaptation](https://mlsanity.com/paper/1903.12230/) (2019)
- [Learning with minibatch Wasserstein : asymptotic and gradient properties](https://mlsanity.com/paper/1910.04091/) (2019)
- [Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation](https://mlsanity.com/paper/2010.05862/) (2020)
- [Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions](https://mlsanity.com/paper/1806.08141/) (2018)
- [Sliced Wasserstein Kernels for Probability Distributions](https://mlsanity.com/paper/1511.03198/) (2015)
- [Adversarial-Learned Loss for Domain Adaptation](https://mlsanity.com/paper/2001.01046/) (2020)
- [Minibatch optimal transport distances; analysis and applications](https://mlsanity.com/paper/2101.01792/) (2021)
- [On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm](https://mlsanity.com/paper/2002.03293/) (2020)
- [Sinkhorn Divergences for Unbalanced Optimal Transport](https://mlsanity.com/paper/1910.12958/) (2019)
- [A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation](https://mlsanity.com/paper/2003.02541/) (2020)
- [Outlier-Robust Optimal Transport](https://mlsanity.com/paper/2012.07363/) (2020)

## Built on by (most cited first, 30 of 125)

- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [SE(3)-Stochastic Flow Matching for Protein Backbone Generation](https://mlsanity.com/paper/2310.02391/) (2023)
- [Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction](https://mlsanity.com/paper/2306.08330/) (2023)
- [Simulation-free Schr"odinger bridges via score and flow matching](https://mlsanity.com/paper/2307.03672/) (2023)
- [LCDNet: Deep Loop Closure Detection and Point Cloud Registration for LiDAR SLAM](https://mlsanity.com/paper/2103.05056/) (2021)
- [Recent Advances in Optimal Transport for Machine Learning](https://mlsanity.com/paper/2306.16156/) (2023)
- [Unified Optimal Transport Framework for Universal Domain Adaptation](https://mlsanity.com/paper/2210.17067/) (2022)
- [DINE: Domain Adaptation from Single and Multiple Black-box Predictors](https://mlsanity.com/paper/2104.01539/) (2021)
- [Unbalanced Optimal Transport, from Theory to Numerics](https://mlsanity.com/paper/2211.08775/) (2022)
- [Global and Local Prompts Cooperation via Optimal Transport for Federated Learning](https://mlsanity.com/paper/2403.00041/) (2024)
- [Improving Mini-batch Optimal Transport via Partial Transportation](https://mlsanity.com/paper/2108.09645/) (2021)
- [Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport](https://mlsanity.com/paper/2305.14777/) (2023)
- [Optimal Transport for Treatment Effect Estimation](https://mlsanity.com/paper/2310.18286/) (2023)
- [Outlier-Robust Optimal Transport: Duality, Structure, and Statistical Analysis](https://mlsanity.com/paper/2111.01361/) (2021)
- [Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey](https://mlsanity.com/paper/2305.05080/) (2023)
- [Addressing Misspecification in Simulation-based Inference through Data-driven Calibration](https://mlsanity.com/paper/2405.08719/) (2024)
- [UMAD: Universal Model Adaptation under Domain and Category Shift](https://mlsanity.com/paper/2112.08553/) (2021)
- [Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe](https://mlsanity.com/paper/2201.00730/) (2022)
- [TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation](https://mlsanity.com/paper/2111.02682/) (2021)
- [SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation](https://mlsanity.com/paper/2304.03251/) (2023)
- [Multi-level Consistency Learning for Semi-supervised Domain Adaptation](https://mlsanity.com/paper/2205.04066/) (2022)
- [Sliced Optimal Partial Transport](https://mlsanity.com/paper/2212.08049/) (2022)
- [Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2109.15286/) (2021)
- [Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors](https://mlsanity.com/paper/2205.14467/) (2022)
- [GenPhys: From Physical Processes to Generative Models](https://mlsanity.com/paper/2304.02637/) (2023)
- [Sobolev Transport: A Scalable Metric for Probability Measures with Graph Metrics](https://mlsanity.com/paper/2202.10723/) (2022)
- [Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings](https://mlsanity.com/paper/2209.15621/) (2022)
- [On Unbalanced Optimal Transport: Gradient Methods, Sparsity and Approximation Error](https://mlsanity.com/paper/2202.03618/) (2022)
- [Unbalanced Optimal Transport: A Unified Framework for Object Detection](https://mlsanity.com/paper/2307.02402/) (2023)
- [k-Mixup Regularization for Deep Learning via Optimal Transport](https://mlsanity.com/paper/2106.02933/) (2021)

## Abstract (arXiv)

Optimal transport distances have found many applications in machine learning for their capacity to compare non-parametric probability distributions. Yet their algorithmic complexity generally prevents their direct use on large scale datasets. Among the possible strategies to alleviate this issue, practitioners can rely on computing estimates of these distances over subsets of data, em i.e. minibatches. While computationally appealing, we highlight in this paper some limits of this strategy, arguing it can lead to undesirable smoothing effects. As an alternative, we suggest that the same minibatch strategy coupled with unbalanced optimal transport can yield more robust behavior. We discuss the associated theoretical properties, such as unbiased estimators, existence of gradients and concentration bounds. Our experimental study shows that in challenging problems associated to domain adaptation, the use of unbalanced optimal transport leads to significantly better results, competing with or surpassing recent baselines.