# Integrative Variational Autoencoders for Generative Modeling of an Image Outcome with Multiple Input Images

arXiv: https://arxiv.org/abs/2402.02734 · HTML: https://mlsanity.com/paper/2402.02734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 70 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 26 of 26)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Recent Advances in Autoencoder-Based Representation Learning](https://mlsanity.com/paper/1812.05069/) (2018)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [Unsupervised speech representation learning using WaveNet autoencoders](https://mlsanity.com/paper/1901.08810/) (2019)
- [Hierarchical Variational Models](https://mlsanity.com/paper/1511.02386/) (2015)
- [Learning Hierarchical Priors in VAEs](https://mlsanity.com/paper/1905.04982/) (2019)
- [A Review on Methods and Applications in Multimodal Deep Learning](https://mlsanity.com/paper/2202.09195/) (2022)
- [Geometric Autoencoders -- What You See is What You Decode](https://mlsanity.com/paper/2306.17638/) (2023)
- [Tensor-on-tensor regression](https://mlsanity.com/paper/1701.01037/) (2017)
- [Generalized Deep Image to Image Regression](https://mlsanity.com/paper/1612.03268/) (2016)
- [TADPOLE Challenge: Accurate Alzheimer's disease prediction through crowdsourced forecasting of future data](https://mlsanity.com/paper/2001.09016/) (2020)
- [Revisiting Structured Variational Autoencoders](https://mlsanity.com/paper/2305.16543/) (2023)
- [Multi-Source Neural Variational Inference](https://mlsanity.com/paper/1811.04451/) (2018)
- [Distributed Bayesian Varying Coefficient Modeling Using a Gaussian Process Prior](https://mlsanity.com/paper/2006.00783/) (2020)
- [Cooperation in the Latent Space: The Benefits of Adding Mixture Components in Variational Autoencoders](https://mlsanity.com/paper/2209.15514/) (2022)
- [Self-Supervised Pre-Training for Deep Image Prior-Based Robust PET Image Denoising](https://mlsanity.com/paper/2302.13546/) (2023)
- [FusionVAE: A Deep Hierarchical Variational Autoencoder for RGB Image Fusion](https://mlsanity.com/paper/2209.11277/) (2022)
- [A tensor based varying-coefficient model for multi-modal neuroimaging data analysis](https://mlsanity.com/paper/2303.16443/) (2023)
- [Unscented Autoencoder](https://mlsanity.com/paper/2306.05256/) (2023)

## Abstract (arXiv)

Understanding relationships across multiple imaging modalities is central to neuroimaging research. We introduce the Integrative Variational Autoencoder (InVA), the first hierarchical VAE framework for image-on-image regression in multimodal neuroimaging. Unlike standard VAEs, which are not designed for predictive integration across modalities, InVA models outcome images as functions of both shared and modality-specific features. This flexible, data-driven approach avoids rigid assumptions of classical tensor regression and outperforms conventional VAEs and nonlinear models such as BART. As a key application, InVA accurately predicts costly PET scans from structural MRI, offering an efficient and powerful tool for multimodal neuroimaging.