# Constraining Variational Inference with Geometric Jensen-Shannon Divergence

arXiv: https://arxiv.org/abs/2006.10599 · HTML: https://mlsanity.com/paper/2006.10599/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 17 of 17)

- [Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation](https://mlsanity.com/paper/2410.13640/) (2024)
- [Trading Off Privacy, Utility and Efficiency in Federated Learning](https://mlsanity.com/paper/2209.00230/) (2022)
- [q-Paths: Generalizing the Geometric Annealing Path using Power Means](https://mlsanity.com/paper/2107.00745/) (2021)
- [Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks](https://mlsanity.com/paper/2209.11366/) (2022)
- [Affinity Clustering Framework for Data Debiasing Using Pairwise Distribution Discrepancy](https://mlsanity.com/paper/2306.01699/) (2023)
- [Text-to-3D Generation using Jensen-Shannon Score Distillation](https://mlsanity.com/paper/2503.10660/) (2025)
- [A Max-relevance-min-divergence Criterion for Data Discretization with Applications on Naive Bayes](https://mlsanity.com/paper/2209.10095/) (2022)
- [Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models](https://mlsanity.com/paper/2407.08843/) (2024)
- [EMPERROR: A Flexible Generative Perception Error Model for Probing Self-Driving Planners](https://mlsanity.com/paper/2411.07719/) (2024)
- [Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators](https://mlsanity.com/paper/2507.14652/) (2025)
- [A Unified Learn-to-Distort-Data Framework for Privacy-Utility Trade-off in Trustworthy Federated Learning](https://mlsanity.com/paper/2407.04751/) (2024)
- [Enhanced uncertainty quantification variational autoencoders for the solution of Bayesian inverse problems](https://mlsanity.com/paper/2502.13105/) (2025)
- [Learning to Score](https://mlsanity.com/paper/2504.14302/) (2025)
- [From Points to Spheres: A Geometric Reinterpretation of Variational Autoencoders](https://mlsanity.com/paper/2507.17255/) (2025)
- [Two tales for a geometric Jensen--Shannon divergence](https://mlsanity.com/paper/2508.05066/) (2025)
- [Surprisal-R'enyi Free Energy](https://mlsanity.com/paper/2603.03405/) (2026)
- [Bayesian 3D Steerable CNNs: Enabling Equivariance and Uncertainty Quantification Simultaneously](https://mlsanity.com/paper/2606.15479/) (2026)

## Abstract (arXiv)

We examine the problem of controlling divergences for latent space regularisation in variational autoencoders. Specifically, when aiming to reconstruct example x∈ℝm via latent space z∈ℝn (n≤ m), while balancing this against the need for generalisable latent representations. We present a regularisation mechanism based on the skew-geometric Jensen-Shannon divergence left(textrmJS^textrmGαright). We find a variation in textrmJS^textrmGα, motivated by limiting cases, which leads to an intuitive interpolation between forward and reverse KL in the space of both distributions and divergences. We motivate its potential benefits for VAEs through low-dimensional examples, before presenting quantitative and qualitative results. Our experiments demonstrate that skewing our variant of textrmJS^textrmGα, in the context of textrmJS^textrmGα-VAEs, leads to better reconstruction and generation when compared to several baseline VAEs. Our approach is entirely unsupervised and utilises only one hyperparameter which can be easily interpreted in latent space.