# Optimal Stability of KL Divergence under Gaussian Perturbations

arXiv: https://arxiv.org/abs/2604.11026 · HTML: https://mlsanity.com/paper/2604.11026/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Why Normalizing Flows Fail to Detect Out-of-Distribution Data](https://mlsanity.com/paper/2006.08545/) (2020)
- [Normalizing Flows are Capable Generative Models](https://mlsanity.com/paper/2412.06329/) (2024)
- [An elementary introduction to information geometry](https://mlsanity.com/paper/1808.08271/) (2018)
- [Constrained Variational Policy Optimization for Safe Reinforcement Learning](https://mlsanity.com/paper/2201.11927/) (2022)
- [Optimal Bounds between f-Divergences and Integral Probability Metrics](https://mlsanity.com/paper/2006.05973/) (2020)
- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [A Less Biased Evaluation of Out-of-distribution Sample Detectors](https://mlsanity.com/paper/1809.04729/) (2018)
- [Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection](https://mlsanity.com/paper/2303.01201/) (2023)
- [Handling Out-of-Distribution Data: A Survey](https://mlsanity.com/paper/2507.21160/) (2025)
- [Online Markov decision processes with Kullback-Leibler control cost](https://mlsanity.com/paper/1401.3198/) (2014)
- [Sparse Coding on Symmetric Positive Definite Manifolds using Bregman Divergences](https://mlsanity.com/paper/1409.0083/) (2014)
- [Are generative deep models for novelty detection truly better?](https://mlsanity.com/paper/1807.05027/) (2018)
- [Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models](https://mlsanity.com/paper/2002.03328/) (2020)
- [Unifying computational entropies via Kullback-Leibler divergence](https://mlsanity.com/paper/1902.11202/) (2019)
- [Out-of-distribution Detection by Cross-class Vicinity Distribution of In-distribution Data](https://mlsanity.com/paper/2206.09385/) (2022)
- [Relaxed Triangle Inequality for Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2602.02577/) (2026)

## Abstract (arXiv)

We study the problem of characterizing the stability of Kullback-Leibler (KL) divergence under Gaussian perturbations beyond Gaussian families. Existing relaxed triangle inequalities for KL divergence critically rely on the assumption that all involved distributions are Gaussian, which limits their applicability in modern applications such as out-of-distribution (OOD) detection with flow-based generative models. In this paper, we remove this restriction by establishing a sharp stability bound between an arbitrary distribution and Gaussian families under mild moment conditions. Specifically, let P be a distribution with finite second moment, and let N1 and N2 be multivariate Gaussian distributions. We show that if KL(P||N1) is large and KL(N1||N2) is at most ε, then KL(P||N2) ≥ KL(P||N1) - O(√ε). Moreover, we prove that this √ε rate is optimal in general, even within the Gaussian family. This result reveals an intrinsic stability property of KL divergence under Gaussian perturbations, extending classical Gaussian-only relaxed triangle inequalities to general distributions. The result is non-trivial due to the asymmetry of KL divergence and the absence of a triangle inequality in general probability spaces. As an application, we provide a rigorous foundation for KL-based OOD analysis in flow-based models, removing strong Gaussian assumptions used in prior work. More broadly, our result enables KL-based reasoning in non-Gaussian settings arising in deep learning and reinforcement learning.