# Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence

arXiv: https://arxiv.org/abs/1908.09211 · HTML: https://mlsanity.com/paper/1908.09211/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2019.

## Built on by (most cited first, 4 of 4)

- [Out-of-distribution Detection and Generation using Soft Brownian Offset Sampling and Autoencoders](https://mlsanity.com/paper/2105.02965/) (2021)
- [On Information (pseudo) Metric](https://mlsanity.com/paper/2103.02008/) (2021)
- [Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation: Bridging KL-Divergence and Integral Probability Metrics](https://mlsanity.com/paper/2201.13127/) (2022)
- [Stable Parallel Training of Wasserstein Conditional Generative Adversarial Neural Networks](https://mlsanity.com/paper/2207.12315/) (2022)

## Abstract (arXiv)

We discuss a relation between the Kantorovich-Wasserstein (KW) metric and the Kullback-Leibler (KL) divergence. The former is defined using the optimal transport problem (OTP) in the Kantorovich formulation. The latter is used to define entropy and mutual information, which appear in variational problems to find optimal channel (OCP) from the rate distortion and the value of information theories. We show that OTP is equivalent to OCP with one additional constraint fixing the output measure, and therefore OCP with constraints on the KL-divergence gives a lower bound on the KW-metric. The dual formulation of OTP allows us to explore the relation between the KL-divergence and the KW-metric using decomposition of the former based on the law of cosines. This way we show the link between two divergences using the variational and geometric principles.