# Discriminative Multimodal Learning via Conditional Priors in Generative Models

arXiv: https://arxiv.org/abs/2110.04616 · HTML: https://mlsanity.com/paper/2110.04616/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 9 Oct 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [InfoVAE: Information Maximizing Variational Autoencoders](https://mlsanity.com/paper/1706.02262/) (2017)
- [Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse](https://mlsanity.com/paper/1911.02469/) (2019)
- [Joint Multimodal Learning with Deep Generative Models](https://mlsanity.com/paper/1611.01891/) (2016)
- [Generalized Multimodal ELBO](https://mlsanity.com/paper/2105.02470/) (2021)
- [Theano: A Python framework for fast computation of mathematical expressions](https://mlsanity.com/paper/1605.02688/) (2016)
- [Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence](https://mlsanity.com/paper/2006.08242/) (2020)
- [Avoiding Latent Variable Collapse With Generative Skip Models](https://mlsanity.com/paper/1807.04863/) (2018)
- [Deep Variational Canonical Correlation Analysis](https://mlsanity.com/paper/1610.03454/) (2016)
- [Generative Models of Visually Grounded Imagination](https://mlsanity.com/paper/1705.10762/) (2017)
- [Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data](https://mlsanity.com/paper/1808.02096/) (2018)
- [Learning Representations by Maximizing Mutual Information in Variational Autoencoders](https://mlsanity.com/paper/1912.13361/) (2019)
- [Deep Generative Models for Reject Inference in Credit Scoring](https://mlsanity.com/paper/1904.11376/) (2019)
- [Multilingual Distributed Representations without Word Alignment](https://mlsanity.com/paper/1312.6173/) (2013)
- [Learning more expressive joint distributions in multimodal variational methods](https://mlsanity.com/paper/2009.03651/) (2020)

## Built on by (most cited first, 4 of 4)

- [Aggregation of Dependent Expert Distributions in Multimodal Variational Autoencoders](https://mlsanity.com/paper/2505.01134/) (2025)
- [Multimodal Generative Models for Bankruptcy Prediction Using Textual Data](https://mlsanity.com/paper/2211.08405/) (2022)
- [Hierarchical mixture of discriminative Generalized Dirichlet classifiers](https://mlsanity.com/paper/2405.01778/) (2024)
- [Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling](https://mlsanity.com/paper/2602.16979/) (2026)

## Abstract (arXiv)

Deep generative models with latent variables have been used lately to learn joint representations and generative processes from multi-modal data. These two learning mechanisms can, however, conflict with each other and representations can fail to embed information on the data modalities. This research studies the realistic scenario in which all modalities and class labels are available for model training, but where some modalities and labels required for downstream tasks are missing. We show, in this scenario, that the variational lower bound limits mutual information between joint representations and missing modalities. We, to counteract these problems, introduce a novel conditional multi-modal discriminative model that uses an informative prior distribution and optimizes a likelihood-free objective function that maximizes mutual information between joint representations and missing modalities. Extensive experimentation demonstrates the benefits of our proposed model, empirical results show that our model achieves state-of-the-art results in representative problems such as downstream classification, acoustic inversion, and image and annotation generation.