# Symmetric Wasserstein Autoencoders

arXiv: https://arxiv.org/abs/2106.13024 · HTML: https://mlsanity.com/paper/2106.13024/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [Max-Sliced Wasserstein Distance and its use for GANs](https://mlsanity.com/paper/1904.05877/) (2019)
- [Distributional Sliced-Wasserstein and Applications to Generative Modeling](https://mlsanity.com/paper/2002.07367/) (2020)
- [RNADE: The real-valued neural autoregressive density-estimator](https://mlsanity.com/paper/1306.0186/) (2013)
- [Structured Disentangled Representations](https://mlsanity.com/paper/1804.02086/) (2018)
- [Learning Hierarchical Priors in VAEs](https://mlsanity.com/paper/1905.04982/) (2019)
- [On the Convergence and Robustness of Training GANs with Regularized Optimal Transport](https://mlsanity.com/paper/1802.08249/) (2018)
- [ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching](https://mlsanity.com/paper/1709.01215/) (2017)
- [Symmetric Variational Autoencoder and Connections to Adversarial Learning](https://mlsanity.com/paper/1709.01846/) (2017)
- [Adversarial Symmetric Variational Autoencoder](https://mlsanity.com/paper/1711.04915/) (2017)
- [The Deep Weight Prior](https://mlsanity.com/paper/1810.06943/) (2018)
- [On the Limitations of First-Order Approximation in GAN Dynamics](https://mlsanity.com/paper/1706.09884/) (2017)
- [Entropic GANs meet VAEs: A Statistical Approach to Compute Sample Likelihoods in GANs](https://mlsanity.com/paper/1810.04147/) (2018)
- [MIM: Mutual Information Machine](https://mlsanity.com/paper/1910.03175/) (2019)

## Abstract (arXiv)

Leveraging the framework of Optimal Transport, we introduce a new family of generative autoencoders with a learnable prior, called Symmetric Wasserstein Autoencoders (SWAEs). We propose to symmetrically match the joint distributions of the observed data and the latent representation induced by the encoder and the decoder. The resulting algorithm jointly optimizes the modelling losses in both the data and the latent spaces with the loss in the data space leading to the denoising effect. With the symmetric treatment of the data and the latent representation, the algorithm implicitly preserves the local structure of the data in the latent space. To further improve the quality of the latent representation, we incorporate a reconstruction loss into the objective, which significantly benefits both the generation and reconstruction. We empirically show the superior performance of SWAEs over the state-of-the-art generative autoencoders in terms of classification, reconstruction, and generation.