# Soft-IntroVAE: Analyzing and Improving the Introspective Variational Autoencoder

arXiv: https://arxiv.org/abs/2012.13253 · HTML: https://mlsanity.com/paper/2012.13253/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2020.

## Built on by (most cited first, 23 of 23)

- [AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensics](https://mlsanity.com/paper/2304.06870/) (2023)
- [Unsupervised Image Representation Learning with Deep Latent Particles](https://mlsanity.com/paper/2205.15821/) (2022)
- [Reversing the Abnormal: Pseudo-Healthy Generative Networks for Anomaly Detection](https://mlsanity.com/paper/2303.08452/) (2023)
- [DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decoder](https://mlsanity.com/paper/2211.11591/) (2022)
- [Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes](https://mlsanity.com/paper/2408.17421/) (2024)
- [What do we learn? Debunking the Myth of Unsupervised Outlier Detection](https://mlsanity.com/paper/2206.03698/) (2022)
- [Towards Universal Unsupervised Anomaly Detection in Medical Imaging](https://mlsanity.com/paper/2401.10637/) (2024)
- [Revisiting Likelihood-Based Out-of-Distribution Detection by Modeling Representations](https://mlsanity.com/paper/2504.07793/) (2025)
- [Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement](https://mlsanity.com/paper/2206.04854/) (2022)
- [AS-IntroVAE: Adversarial Similarity Distance Makes Robust IntroVAE](https://mlsanity.com/paper/2206.13903/) (2022)
- [Fine-Tuning Generative Models as an Inference Method for Robotic Tasks](https://mlsanity.com/paper/2310.12862/) (2023)
- [Towards Controllable Time Series Generation](https://mlsanity.com/paper/2403.03698/) (2024)
- [Multimodal ELBO with Diffusion Decoders](https://mlsanity.com/paper/2408.16883/) (2024)
- [Improving Deep Learning-based Automatic Cranial Defect Reconstruction by Heavy Data Augmentation: From Image Registration to Latent Diffusion Models](https://mlsanity.com/paper/2406.06372/) (2024)
- [Interpretable Representation Learning of Cardiac MRI via Attribute Regularization](https://mlsanity.com/paper/2406.08282/) (2024)
- [Selective Test-Time Adaptation for Unsupervised Anomaly Detection using Neural Implicit Representations](https://mlsanity.com/paper/2410.03306/) (2024)
- [A DeepONet joint Neural Tangent Kernel Hybrid Framework for Physics-Informed Inverse Source Problems and Robust Image Reconstruction](https://mlsanity.com/paper/2511.00338/) (2025)
- [Quantum Generative Diffusion Model for Real-World Time Series](https://mlsanity.com/paper/2606.27561/) (2026)
- [Soft-IntroVAE for Continuous Latent space Image Super-Resolution](https://mlsanity.com/paper/2307.09008/) (2023)
- [Attribute Regularized Soft Introspective VAE: Towards Cardiac Attribute Regularization Through MRI Domains](https://mlsanity.com/paper/2307.12618/) (2023)
- [Attribute Regularized Soft Introspective Variational Autoencoder for Interpretable Cardiac Disease Classification](https://mlsanity.com/paper/2312.08915/) (2023)
- [Multimodal normative modeling in Alzheimers Disease with introspective variational autoencoders](https://mlsanity.com/paper/2602.08077/) (2026)
- [Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI](https://mlsanity.com/paper/2608.16725/) (2026)

## Abstract (arXiv)

The recently introduced introspective variational autoencoder (IntroVAE) exhibits outstanding image generations, and allows for amortized inference using an image encoder. The main idea in IntroVAE is to train a VAE adversarially, using the VAE encoder to discriminate between generated and real data samples. However, the original IntroVAE loss function relied on a particular hinge-loss formulation that is very hard to stabilize in practice, and its theoretical convergence analysis ignored important terms in the loss. In this work, we take a step towards better understanding of the IntroVAE model, its practical implementation, and its applications. We propose the Soft-IntroVAE, a modified IntroVAE that replaces the hinge-loss terms with a smooth exponential loss on generated samples. This change significantly improves training stability, and also enables theoretical analysis of the complete algorithm. Interestingly, we show that the IntroVAE converges to a distribution that minimizes a sum of KL distance from the data distribution and an entropy term. We discuss the implications of this result, and demonstrate that it induces competitive image generation and reconstruction. Finally, we describe two applications of Soft-IntroVAE to unsupervised image translation and out-of-distribution detection, and demonstrate compelling results. Code and additional information is available on the project website -- https://taldatech.github.io/soft-intro-vae-web