# Information Constraints on Auto-Encoding Variational Bayes

arXiv: https://arxiv.org/abs/1805.08672 · HTML: https://mlsanity.com/paper/1805.08672/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 30 of 33)

- [Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspective](https://mlsanity.com/paper/2106.07115/) (2021)
- [Geometry Aligned Variational Transformer for Image-conditioned Layout Generation](https://mlsanity.com/paper/2209.00852/) (2022)
- [Moment Matching Deep Contrastive Latent Variable Models](https://mlsanity.com/paper/2202.10560/) (2022)
- [ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splitting](https://mlsanity.com/paper/2205.13700/) (2022)
- [Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent](https://mlsanity.com/paper/2502.11740/) (2025)
- [Disentangled Spatiotemporal Graph Generative Models](https://mlsanity.com/paper/2203.00411/) (2022)
- [Long-term Causal Effects Estimation via Latent Surrogates Representation Learning](https://mlsanity.com/paper/2208.04589/) (2022)
- [InfoVAEGAN : learning joint interpretable representations by information maximization and maximum likelihood](https://mlsanity.com/paper/2107.04705/) (2021)
- [Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models](https://mlsanity.com/paper/2412.05538/) (2024)
- [DisUnknown: Distilling Unknown Factors for Disentanglement Learning](https://mlsanity.com/paper/2109.08090/) (2021)
- [Leveraging Relational Information for Learning Weakly Disentangled Representations](https://mlsanity.com/paper/2205.10056/) (2022)
- [AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Data](https://mlsanity.com/paper/2503.05665/) (2025)
- [Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval](https://mlsanity.com/paper/2103.12328/) (2021)
- [Unsupervised Disentanglement without Autoencoding: Pitfalls and Future Directions](https://mlsanity.com/paper/2108.06613/) (2021)
- [Toward the Identifiability of Comparative Deep Generative Models](https://mlsanity.com/paper/2401.15903/) (2024)
- [Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness](https://mlsanity.com/paper/2106.08161/) (2021)
- [Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints](https://mlsanity.com/paper/2203.15155/) (2022)
- [A Survey: Spatiotemporal Consistency in Video Generation](https://mlsanity.com/paper/2502.17863/) (2025)
- [GPTDrawer: Enhancing Visual Synthesis through ChatGPT](https://mlsanity.com/paper/2412.10429/) (2024)
- [Supervised Contrastive Block Disentanglement](https://mlsanity.com/paper/2502.07281/) (2025)
- [Unsupervised Representation Learning - an Invariant Risk Minimization Perspective](https://mlsanity.com/paper/2505.12506/) (2025)
- [Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder](https://mlsanity.com/paper/2506.13658/) (2025)
- [SC-GIR: Goal-oriented Semantic Communication via Invariant Representation Learning](https://mlsanity.com/paper/2509.01119/) (2025)
- [Variational Auto-Encoder Architectures that Excel at Causal Inference](https://mlsanity.com/paper/2111.06486/) (2021)
- [Causal Effect Estimation using identifiable Variational AutoEncoder with Latent Confounders and Post-Treatment Variables](https://mlsanity.com/paper/2408.07219/) (2024)
- [Disentangling Interpretable Factors with Supervised Independent Subspace Principal Component Analysis](https://mlsanity.com/paper/2410.23595/) (2024)
- [A Machine Learning Approach for Denoising and Upsampling HRTFs](https://mlsanity.com/paper/2504.17586/) (2025)
- [Permutation-Free High-Order Interaction Tests](https://mlsanity.com/paper/2506.05963/) (2025)
- [Disentangling semantic features of macromolecules in Cryo-Electron Tomography](https://mlsanity.com/paper/2106.14192/) (2021)
- [Synchronous Image-Label Diffusion Probability Model with Application to Stroke Lesion Segmentation on Non-contrast CT](https://mlsanity.com/paper/2307.01740/) (2023)

## Abstract (arXiv)

Parameterizing the approximate posterior of a generative model with neural networks has become a common theme in recent machine learning research. While providing appealing flexibility, this approach makes it difficult to impose or assess structural constraints such as conditional independence. We propose a framework for learning representations that relies on Auto-Encoding Variational Bayes and whose search space is constrained via kernel-based measures of independence. In particular, our method employs the d-variable Hilbert-Schmidt Independence Criterion (dHSIC) to enforce independence between the latent representations and arbitrary nuisance factors. We show how to apply this method to a range of problems, including the problems of learning invariant representations and the learning of interpretable representations. We also present a full-fledged application to single-cell RNA sequencing (scRNA-seq). In this setting the biological signal is mixed in complex ways with sequencing errors and sampling effects. We show that our method out-performs the state-of-the-art in this domain.