# Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance

arXiv: https://arxiv.org/abs/1805.11897 · HTML: https://mlsanity.com/paper/1805.11897/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2018.

## Built on by (most cited first, 30 of 56)

- [Sinkformers: Transformers with Doubly Stochastic Attention](https://mlsanity.com/paper/2110.11773/) (2021)
- [Optimal Transport Tools (OTT): A JAX Toolbox for all things Wasserstein](https://mlsanity.com/paper/2201.12324/) (2022)
- [Recent Advances in Optimal Transport for Machine Learning](https://mlsanity.com/paper/2306.16156/) (2023)
- [Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections](https://mlsanity.com/paper/2107.01718/) (2021)
- [Sinkhorn Distributionally Robust Optimization](https://mlsanity.com/paper/2109.11926/) (2021)
- [Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach](https://mlsanity.com/paper/2106.02968/) (2021)
- [WaterNeRF: Neural Radiance Fields for Underwater Scenes](https://mlsanity.com/paper/2209.13091/) (2022)
- [A Unified Framework for Implicit Sinkhorn Differentiation](https://mlsanity.com/paper/2205.06688/) (2022)
- [One-step differentiation of iterative algorithms](https://mlsanity.com/paper/2305.13768/) (2023)
- [Learning to Predict Graphs with Fused Gromov-Wasserstein Barycenters](https://mlsanity.com/paper/2202.03813/) (2022)
- [Bi-level Feature Alignment for Versatile Image Translation and Manipulation](https://mlsanity.com/paper/2107.03021/) (2021)
- [ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgery](https://mlsanity.com/paper/2112.08189/) (2021)
- [Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference](https://mlsanity.com/paper/2107.12270/) (2021)
- [Learning Generalized Gumbel-max Causal Mechanisms](https://mlsanity.com/paper/2111.06888/) (2021)
- [Rethinking Initialization of the Sinkhorn Algorithm](https://mlsanity.com/paper/2206.07630/) (2022)
- [Accelerating Sinkhorn Algorithm with Sparse Newton Iterations](https://mlsanity.com/paper/2401.12253/) (2024)
- [Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization](https://mlsanity.com/paper/2211.07096/) (2022)
- [The derivatives of Sinkhorn-Knopp converge](https://mlsanity.com/paper/2207.12717/) (2022)
- [Convergence of Gaussian-smoothed optimal transport distance with sub-gamma distributions and dependent samples](https://mlsanity.com/paper/2103.00394/) (2021)
- [An Exploration of Learnt Representations of W Jets](https://mlsanity.com/paper/2109.10919/) (2021)
- [KNOT: Knowledge Distillation using Optimal Transport for Solving NLP Tasks](https://mlsanity.com/paper/2110.02432/) (2021)
- [Contrasting Multiple Representations with the Multi-Marginal Matching Gap](https://mlsanity.com/paper/2405.19532/) (2024)
- [FGOT: Graph Distances based on Filters and Optimal Transport](https://mlsanity.com/paper/2109.04442/) (2021)
- [VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priors](https://mlsanity.com/paper/2405.02962/) (2024)
- [A New Robust Partial p-Wasserstein-Based Metric for Comparing Distributions](https://mlsanity.com/paper/2405.03664/) (2024)
- [Exploiting Problem Structure in Deep Declarative Networks: Two Case Studies](https://mlsanity.com/paper/2202.12404/) (2022)
- [Learning Graphon Autoencoders for Generative Graph Modeling](https://mlsanity.com/paper/2105.14244/) (2021)
- [Consistency of Neural Causal Partial Identification](https://mlsanity.com/paper/2405.15673/) (2024)
- [Optimal transport with f-divergence regularization and generalized Sinkhorn algorithm](https://mlsanity.com/paper/2105.14337/) (2021)
- [Statistical and Computational Guarantees of Kernel Max-Sliced Wasserstein Distances](https://mlsanity.com/paper/2405.15441/) (2024)

## Abstract (arXiv)

Applications of optimal transport have recently gained remarkable attention thanks to the computational advantages of entropic regularization. However, in most situations the Sinkhorn approximation of the Wasserstein distance is replaced by a regularized version that is less accurate but easy to differentiate. In this work we characterize the differential properties of the original Sinkhorn distance, proving that it enjoys the same smoothness as its regularized version and we explicitly provide an efficient algorithm to compute its gradient. We show that this result benefits both theory and applications: on one hand, high order smoothness confers statistical guarantees to learning with Wasserstein approximations. On the other hand, the gradient formula allows us to efficiently solve learning and optimization problems in practice. Promising preliminary experiments complement our analysis.