# Disentangling shared and private latent factors in multimodal Variational Autoencoders

arXiv: https://arxiv.org/abs/2403.06338 · HTML: https://mlsanity.com/paper/2403.06338/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 10 Mar 2024.

## Builds on (most cited first, 8 of 8)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [Joint Multimodal Learning with Deep Generative Models](https://mlsanity.com/paper/1611.01891/) (2016)
- [Generalized Multimodal ELBO](https://mlsanity.com/paper/2105.02470/) (2021)
- [Deep Variational Canonical Correlation Analysis](https://mlsanity.com/paper/1610.03454/) (2016)
- [A Variational Information Bottleneck Approach to Multi-Omics Data Integration](https://mlsanity.com/paper/2102.03014/) (2021)
- [Multi-view Learning as a Nonparametric Nonlinear Inter-Battery Factor Analysis](https://mlsanity.com/paper/1604.04939/) (2016)

## Built on by (most cited first, 3 of 3)

- [MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation](https://mlsanity.com/paper/2606.16408/) (2026)
- [Disentangling Shared and Target-Enriched Topics via Background-Contrastive Non-negative Matrix Factorization](https://mlsanity.com/paper/2602.22387/) (2026)
- [Disentangled Shared Representations Improve Morpho-Transcriptomic Integration](https://mlsanity.com/paper/2608.14355/) (2026)

## Abstract (arXiv)

Generative models for multimodal data permit the identification of latent factors that may be associated with important determinants of observed data heterogeneity. Common or shared factors could be important for explaining variation across modalities whereas other factors may be private and important only for the explanation of a single modality. Multimodal Variational Autoencoders, such as MVAE and MMVAE, are a natural choice for inferring those underlying latent factors and separating shared variation from private. In this work, we investigate their capability to reliably perform this disentanglement. In particular, we highlight a challenging problem setting where modality-specific variation dominates the shared signal. Taking a cross-modal prediction perspective, we demonstrate limitations of existing models, and propose a modification how to make them more robust to modality-specific variation. Our findings are supported by experiments on synthetic as well as various real-world multi-omics data sets.