# Multi-Source Neural Variational Inference

arXiv: https://arxiv.org/abs/1811.04451 · HTML: https://mlsanity.com/paper/1811.04451/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2018.

## Built on by (most cited first, 9 of 9)

- [Generalized Multimodal ELBO](https://mlsanity.com/paper/2105.02470/) (2021)
- [On the Limitations of Multimodal VAEs](https://mlsanity.com/paper/2110.04121/) (2021)
- [Representation Uncertainty in Self-Supervised Learning as Variational Inference](https://mlsanity.com/paper/2203.11437/) (2022)
- [Reconstructing Nonlinear Dynamical Systems from Multi-Modal Time Series](https://mlsanity.com/paper/2111.02922/) (2021)
- [Continual Variational Autoencoder Learning via Online Cooperative Memorization](https://mlsanity.com/paper/2207.10131/) (2022)
- [Lifelong Mixture of Variational Autoencoders](https://mlsanity.com/paper/2107.04694/) (2021)
- [Multimodal Adversarially Learned Inference with Factorized Discriminators](https://mlsanity.com/paper/2112.10384/) (2021)
- [Learning Mixture Structure on Multi-Source Time Series for Probabilistic Forecasting](https://mlsanity.com/paper/2302.11078/) (2023)
- [Integrative Variational Autoencoders for Generative Modeling of an Image Outcome with Multiple Input Images](https://mlsanity.com/paper/2402.02734/) (2024)

## Abstract (arXiv)

Learning from multiple sources of information is an important problem in machine-learning research. The key challenges are learning representations and formulating inference methods that take into account the complementarity and redundancy of various information sources. In this paper we formulate a variational autoencoder based multi-source learning framework in which each encoder is conditioned on a different information source. This allows us to relate the sources via the shared latent variables by computing divergence measures between individual source's posterior approximations. We explore a variety of options to learn these encoders and to integrate the beliefs they compute into a consistent posterior approximation. We visualise learned beliefs on a toy dataset and evaluate our methods for learning shared representations and structured output prediction, showing trade-offs of learning separate encoders for each information source. Furthermore, we demonstrate how conflict detection and redundancy can increase robustness of inference in a multi-source setting.