# Adversarial robustness of VAEs through the lens of local geometry

arXiv: https://arxiv.org/abs/2208.03923 · HTML: https://mlsanity.com/paper/2208.03923/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction](https://mlsanity.com/paper/2103.04174/) (2021)
- [Structured Disentangled Representations](https://mlsanity.com/paper/1804.02086/) (2018)
- [Adversarial examples for generative models](https://mlsanity.com/paper/1702.06832/) (2017)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy](https://mlsanity.com/paper/1906.06784/) (2019)
- [Geometrically Enriched Latent Spaces](https://mlsanity.com/paper/2008.00565/) (2020)
- [Learning Flat Latent Manifolds with VAEs](https://mlsanity.com/paper/2002.04881/) (2020)
- [Adversarial Attacks Against Deep Generative Models on Data: A Survey](https://mlsanity.com/paper/2112.00247/) (2021)
- [Adversarial Images for Variational Autoencoders](https://mlsanity.com/paper/1612.00155/) (2016)
- [Adversarial Attacks on Variational Autoencoders](https://mlsanity.com/paper/1806.04646/) (2018)
- [The Adversarial Attack and Detection under the Fisher Information Metric](https://mlsanity.com/paper/1810.03806/) (2018)
- [Fisher-Bures Adversary Graph Convolutional Networks](https://mlsanity.com/paper/1903.04154/) (2019)
- [Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)](https://mlsanity.com/paper/1806.04994/) (2018)
- [Towards a Theoretical Understanding of the Robustness of Variational Autoencoders](https://mlsanity.com/paper/2007.07365/) (2020)
- [Geodesic Clustering in Deep Generative Models](https://mlsanity.com/paper/1809.04747/) (2018)
- [Alleviating Adversarial Attacks on Variational Autoencoders with MCMC](https://mlsanity.com/paper/2203.09940/) (2022)
- [Expected path length on random manifolds](https://mlsanity.com/paper/1908.07377/) (2019)

## Built on by (most cited first, 4 of 4)

- [Weakly Augmented Variational Autoencoder in Time Series Anomaly Detection](https://mlsanity.com/paper/2401.03341/) (2024)
- [Ricci flow regularization in latent spaces for the forward learning of partial differential equations](https://mlsanity.com/paper/2401.14591/) (2024)
- [LatentStealth: Unnoticeable and Efficient Adversarial Attacks on Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2505.12009/) (2025)
- [Commutator-Induced Uncertainty in VAEs](https://mlsanity.com/paper/2605.23449/) (2026)

## Abstract (arXiv)

In an unsupervised attack on variational autoencoders (VAEs), an adversary finds a small perturbation in an input sample that significantly changes its latent space encoding, thereby compromising the reconstruction for a fixed decoder. A known reason for such vulnerability is the distortions in the latent space resulting from a mismatch between approximated latent posterior and a prior distribution. Consequently, a slight change in an input sample can move its encoding to a low/zero density region in the latent space resulting in an unconstrained generation. This paper demonstrates that an optimal way for an adversary to attack VAEs is to exploit a directional bias of a stochastic pullback metric tensor induced by the encoder and decoder networks. The pullback metric tensor of an encoder measures the change in infinitesimal latent volume from an input to a latent space. Thus, it can be viewed as a lens to analyse the effect of input perturbations leading to latent space distortions. We propose robustness evaluation scores using the eigenspectrum of a pullback metric tensor. Moreover, we empirically show that the scores correlate with the robustness parameter β of the β-VAE. Since increasing β also degrades reconstruction quality, we demonstrate a simple alternative using textitmixup training to fill the empty regions in the latent space, thus improving robustness with improved reconstruction.