# Multimodal Adversarially Learned Inference with Factorized Discriminators

arXiv: https://arxiv.org/abs/2112.10384 · HTML: https://mlsanity.com/paper/2112.10384/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 20 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [cGANs with Projection Discriminator](https://mlsanity.com/paper/1802.05637/) (2018)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [Learning Factorized Multimodal Representations](https://mlsanity.com/paper/1806.06176/) (2018)
- [Variational Autoencoder for Deep Learning of Images, Labels and Captions](https://mlsanity.com/paper/1609.08976/) (2016)
- [Joint Multimodal Learning with Deep Generative Models](https://mlsanity.com/paper/1611.01891/) (2016)
- [Generalized Multimodal ELBO](https://mlsanity.com/paper/2105.02470/) (2021)
- [The Unusual Effectiveness of Averaging in GAN Training](https://mlsanity.com/paper/1806.04498/) (2018)
- [Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence](https://mlsanity.com/paper/2006.08242/) (2020)
- [Generative Models of Visually Grounded Imagination](https://mlsanity.com/paper/1705.10762/) (2017)
- [ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching](https://mlsanity.com/paper/1709.01215/) (2017)
- [Relating by Contrasting: A Data-efficient Framework for Multimodal Generative Models](https://mlsanity.com/paper/2007.01179/) (2020)
- [Multi-Source Neural Variational Inference](https://mlsanity.com/paper/1811.04451/) (2018)
- [Visual Dialogue without Vision or Dialogue](https://mlsanity.com/paper/1812.06417/) (2018)
- [Variational Inference via chi-Upper Bound Minimization](https://mlsanity.com/paper/1611.00328/) (2016)
- [Generalized Adversarially Learned Inference](https://mlsanity.com/paper/2006.08089/) (2020)
- [Training Generative Adversarial Networks from Incomplete Observations using Factorised Discriminators](https://mlsanity.com/paper/1905.12660/) (2019)

## Built on by (most cited first, 2 of 2)

- [Text-to-Image Cross-Modal Generation: A Systematic Review](https://mlsanity.com/paper/2401.11631/) (2024)
- [Aggregation of Dependent Expert Distributions in Multimodal Variational Autoencoders](https://mlsanity.com/paper/2505.01134/) (2025)

## Abstract (arXiv)

Learning from multimodal data is an important research topic in machine learning, which has the potential to obtain better representations. In this work, we propose a novel approach to generative modeling of multimodal data based on generative adversarial networks. To learn a coherent multimodal generative model, we show that it is necessary to align different encoder distributions with the joint decoder distribution simultaneously. To this end, we construct a specific form of the discriminator to enable our model to utilize data efficiently, which can be trained constrastively. By taking advantage of contrastive learning through factorizing the discriminator, we train our model on unimodal data. We have conducted experiments on the benchmark datasets, whose promising results show that our proposed approach outperforms the-state-of-the-art methods on a variety of metrics. The source code will be made publicly available.