# Iterative Alignment Flows

arXiv: https://arxiv.org/abs/2104.07232 · HTML: https://mlsanity.com/paper/2104.07232/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 27 of 27)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1803.10081/) (2018)
- [Generalized Sliced Wasserstein Distances](https://mlsanity.com/paper/1902.00434/) (2019)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Generative Modeling using the Sliced Wasserstein Distance](https://mlsanity.com/paper/1803.11188/) (2018)
- [Sliced Wasserstein Kernels for Probability Distributions](https://mlsanity.com/paper/1511.03198/) (2015)
- [Tree-Sliced Variants of Wasserstein Distances](https://mlsanity.com/paper/1902.00342/) (2019)
- [Continuous Wasserstein-2 Barycenter Estimation without Minimax Optimization](https://mlsanity.com/paper/2102.01752/) (2021)
- [Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks](https://mlsanity.com/paper/2007.04462/) (2020)
- [Sliced Iterative Normalizing Flows](https://mlsanity.com/paper/2007.00674/) (2020)
- [AlignFlow: Cycle Consistent Learning from Multiple Domains via Normalizing Flows](https://mlsanity.com/paper/1905.12892/) (2019)
- [Continuous Regularized Wasserstein Barycenters](https://mlsanity.com/paper/2008.12534/) (2020)
- [Iterative Gaussianization: from ICA to Random Rotations](https://mlsanity.com/paper/1602.00229/) (2016)
- [A Large-Scale Study on Regularization and Normalization in GANs](https://mlsanity.com/paper/1807.04720/) (2018)
- [Log-Likelihood Ratio Minimizing Flows: Towards Robust and Quantifiable Neural Distribution Alignment](https://mlsanity.com/paper/2003.12170/) (2020)
- [Tree-Wasserstein Barycenter for Large-Scale Multilevel Clustering and Scalable Bayes](https://mlsanity.com/paper/1910.04483/) (2019)
- [Adaptive Optimal Transport](https://mlsanity.com/paper/1807.00393/) (2018)

## Built on by (most cited first, 4 of 4)

- [Counterfactual Fairness by Combining Factual and Counterfactual Predictions](https://mlsanity.com/paper/2409.01977/) (2024)
- [Wasserstein Geodesic Generator for Conditional Distributions](https://mlsanity.com/paper/2308.10145/) (2023)
- [Cooperative Distribution Alignment via JSD Upper Bound](https://mlsanity.com/paper/2207.02286/) (2022)
- [Adversarial Learning for Feature Shift Detection and Correction](https://mlsanity.com/paper/2312.04546/) (2023)

## Abstract (arXiv)

The unsupervised task of aligning two or more distributions in a shared latent space has many applications including fair representations, batch effect mitigation, and unsupervised domain adaptation. Existing flow-based approaches estimate multiple flows independently, which is equivalent to learning multiple full generative models. Other approaches require adversarial learning, which can be computationally expensive and challenging to optimize. Thus, we aim to jointly align multiple distributions while avoiding adversarial learning. Inspired by efficient alignment algorithms from optimal transport (OT) theory for univariate distributions, we develop a simple iterative method to build deep and expressive flows. Our method decouples each iteration into two subproblems: 1) form a variational approximation of a distribution divergence and 2) minimize this variational approximation via closed-form invertible alignment maps based on known OT results. Our empirical results give evidence that this iterative algorithm achieves competitive distribution alignment at low computational cost while being able to naturally handle more than two distributions.