# Capturing Label Characteristics in VAEs

arXiv: https://arxiv.org/abs/2006.10102 · HTML: https://mlsanity.com/paper/2006.10102/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 30 of 30)

- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming](https://mlsanity.com/paper/2202.04178/) (2022)
- [MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularization](https://mlsanity.com/paper/2203.14316/) (2022)
- [Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo](https://mlsanity.com/paper/2202.04599/) (2022)
- [Learning Multimodal VAEs through Mutual Supervision](https://mlsanity.com/paper/2106.12570/) (2021)
- [Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms](https://mlsanity.com/paper/2306.01213/) (2023)
- [InfoNCE is variational inference in a recognition parameterised model](https://mlsanity.com/paper/2107.02495/) (2021)
- [SepVAE: a contrastive VAE to separate pathological patterns from healthy ones](https://mlsanity.com/paper/2307.06206/) (2023)
- [Provable concept learning for interpretable predictions using variational autoencoders](https://mlsanity.com/paper/2204.00492/) (2022)
- [Incorporating Unlabelled Data into Bayesian Neural Networks](https://mlsanity.com/paper/2304.01762/) (2023)
- [Disentanglement and Generalization Under Correlation Shifts](https://mlsanity.com/paper/2112.14754/) (2021)
- [Transductive Decoupled Variational Inference for Few-Shot Classification](https://mlsanity.com/paper/2208.10559/) (2022)
- [Double InfoGAN for Contrastive Analysis](https://mlsanity.com/paper/2401.17776/) (2024)
- [Towards Controllable Time Series Generation](https://mlsanity.com/paper/2403.03698/) (2024)
- [Material Microstructure Design Using VAE-Regression with Multimodal Prior](https://mlsanity.com/paper/2402.17806/) (2024)
- [AIRIVA: A Deep Generative Model of Adaptive Immune Repertoires](https://mlsanity.com/paper/2304.13737/) (2023)
- [Automatic Discovery of Disease Subgroups by Contrasting with Healthy Controls](https://mlsanity.com/paper/2605.21301/) (2026)
- [Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility](https://mlsanity.com/paper/2109.04561/) (2021)
- [Disentangled representations: towards interpretation of sex determination from hip bone](https://mlsanity.com/paper/2112.09414/) (2021)
- [The Transitive Information Theory and its Application to Deep Generative Models](https://mlsanity.com/paper/2203.05074/) (2022)
- [Repairing Systematic Outliers by Learning Clean Subspaces in VAEs](https://mlsanity.com/paper/2207.08050/) (2022)
- [Censor Dependent Variational Inference](https://mlsanity.com/paper/2502.09591/) (2025)
- [Variational Learning of Disentangled Representations](https://mlsanity.com/paper/2506.17182/) (2025)
- [Crafting Imperceptible On-Manifold Adversarial Attacks for Tabular Data](https://mlsanity.com/paper/2507.10998/) (2025)
- [Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations](https://mlsanity.com/paper/2202.12932/) (2022)
- [Learning Generative Factors of EEG Data with Variational auto-encoders](https://mlsanity.com/paper/2206.01939/) (2022)
- [Rethinking the Value of Labels for Instance-Dependent Label Noise Learning](https://mlsanity.com/paper/2305.06247/) (2023)
- [Active Learning with Task-Driven Representations for Messy Pools](https://mlsanity.com/paper/2510.25926/) (2025)
- [A Systematic Framework for Tabular Data Disentanglement](https://mlsanity.com/paper/2604.07940/) (2026)
- [Environment-Robust Representation Learning with Empirical Bayes](https://mlsanity.com/paper/2606.05365/) (2026)

## Abstract (arXiv)

We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels. While prior work has typically conflated these by learning latent variables that directly correspond to label values, we argue this is contrary to the intended effect of supervision in VAEs-capturing rich label characteristics with the latents. For example, we may want to capture the characteristics of a face that make it look young, rather than just the age of the person. To this end, we develop the CCVAE, a novel VAE model and concomitant variational objective which captures label characteristics explicitly in the latent space, eschewing direct correspondences between label values and latents. Through judicious structuring of mappings between such characteristic latents and labels, we show that the CCVAE can effectively learn meaningful representations of the characteristics of interest across a variety of supervision schemes. In particular, we show that the CCVAE allows for more effective and more general interventions to be performed, such as smooth traversals within the characteristics for a given label, diverse conditional generation, and transferring characteristics across datapoints.