# Score-Based Multimodal Autoencoder

arXiv: https://arxiv.org/abs/2305.15708 · HTML: https://mlsanity.com/paper/2305.15708/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- First posted on arXiv 25 May 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Built on by (most cited first, 7 of 7)

- [Multi-modal Latent Diffusion](https://mlsanity.com/paper/2306.04445/) (2023)
- [Cohort-Based Active Modality Acquisition](https://mlsanity.com/paper/2505.16791/) (2025)
- [Multimodal ELBO with Diffusion Decoders](https://mlsanity.com/paper/2408.16883/) (2024)
- [Hellinger Multimodal Variational Autoencoders](https://mlsanity.com/paper/2601.06572/) (2026)
- [Graph Representation Learning with Diffusion Generative Models](https://mlsanity.com/paper/2501.13133/) (2025)
- [Learning Multimodal Energy-Based Model with Multimodal Variational Auto-Encoder via MCMC Revision](https://mlsanity.com/paper/2605.00644/) (2026)
- [H"older++: Improving the Quality-Coherence Trade-off in Multimodal VAEs](https://mlsanity.com/paper/2606.13381/) (2026)

## Abstract (arXiv)

Multimodal Variational Autoencoders (VAEs) represent a promising group of generative models that facilitate the construction of a tractable posterior within the latent space given multiple modalities. Previous studies have shown that as the number of modalities increases, the generative quality of each modality declines. In this study, we explore an alternative approach to enhance the generative performance of multimodal VAEs by jointly modeling the latent space of independently trained unimodal VAEs using score-based models (SBMs). The role of the SBM is to enforce multimodal coherence by learning the correlation among the latent variables. Consequently, our model combines a better generative quality of unimodal VAEs with coherent integration across different modalities using the latent score-based model. In addition, our approach provides the best unconditional coherence.