# DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passing

arXiv: https://arxiv.org/abs/2608.02769 · HTML: https://mlsanity.com/paper/2608.02769/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 91 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 30 of 34)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [A modern maximum-likelihood theory for high-dimensional logistic regression](https://mlsanity.com/paper/1803.06964/) (2018)
- [A unifying tutorial on Approximate Message Passing](https://mlsanity.com/paper/2105.02180/) (2021)
- [Estimation of Low-Rank Matrices via Approximate Message Passing](https://mlsanity.com/paper/1711.01682/) (2017)
- [Graph-based Approximate Message Passing Iterations](https://mlsanity.com/paper/2109.11905/) (2021)
- [Angle-Based Joint and Individual Variation Explained](https://mlsanity.com/paper/1704.02060/) (2017)
- [A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications](https://mlsanity.com/paper/2408.02686/) (2024)
- [Approximate Message Passing with Spectral Initialization for Generalized Linear Models](https://mlsanity.com/paper/2010.03460/) (2020)
- [Finite Sample Analysis of Approximate Message Passing Algorithms](https://mlsanity.com/paper/1606.01800/) (2016)
- [Structural Learning and Integrative Decomposition of Multi-View Data](https://mlsanity.com/paper/1707.06573/) (2017)
- [Cooperative learning for multiview analysis](https://mlsanity.com/paper/2112.12337/) (2021)
- [Solving Empirical Bayes via Transformers](https://mlsanity.com/paper/2502.09844/) (2025)
- [Information-theoretic limits of a multiview low-rank symmetric spiked matrix model](https://mlsanity.com/paper/2005.08017/) (2020)
- [Stein's unbiased risk estimate and Hyv"arinen's score matching](https://mlsanity.com/paper/2502.20123/) (2025)
- [Mutual Information in Community Detection with Covariate Information and Correlated Networks](https://mlsanity.com/paper/1912.05375/) (2019)
- [Data Integration Via Analysis of Subspaces (DIVAS)](https://mlsanity.com/paper/2212.00703/) (2022)
- [Fundamental limits of community detection from multi-view data: multi-layer, dynamic and partially labeled block models](https://mlsanity.com/paper/2401.08167/) (2024)
- [Estimating shared subspace with AJIVE: the power and limitation of multiple data matrices](https://mlsanity.com/paper/2501.09336/) (2025)
- [Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variables](https://mlsanity.com/paper/2505.12473/) (2025)
- [Approximate Message Passing for the Matrix Tensor Product Model](https://mlsanity.com/paper/2306.15580/) (2023)
- [Optimal Estimation of Shared Singular Subspaces across Multiple Noisy Matrices](https://mlsanity.com/paper/2411.17054/) (2024)
- [Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model](https://mlsanity.com/paper/2506.02664/) (2025)
- [Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration](https://mlsanity.com/paper/2507.22170/) (2025)
- [Measurement error models: from nonparametric methods to deep neural networks](https://mlsanity.com/paper/2007.07498/) (2020)
- [Bayesian Joint Additive Factor Models for Multiview Learning](https://mlsanity.com/paper/2406.00778/) (2024)
- [Multimodal data integration and cross-modal querying via orchestrated approximate message passing](https://mlsanity.com/paper/2407.19030/) (2024)
- [Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions](https://mlsanity.com/paper/2407.03522/) (2024)
- [Generalizing Variational Autoencoders with Hierarchical Empirical Bayes](https://mlsanity.com/paper/2007.10389/) (2020)

## Abstract (arXiv)

Multimodal supervised learning seeks to leverage multiple heterogeneous data sources to improve predictive performance. A central challenge is determining the fusion granularity across modalities: over-integration may amplify noise while under-integration fails to exploit cross-modal dependence. Existing approaches rely on pre-specified fusion architectures, from early to late fusion, that may not adapt to the underlying dependence structure among modalities. We propose DAIF, a data adaptive intermediate fusion framework that combines random matrix theory and non-parametric dependence measures to learn fusion structure directly from data. We operate under a Bayesian multimodal factor model where the prior on the latent factors determines the cross-modal dependence. Our method clusters modalities based on estimated intermodal dependence, then performs clusterwise empirical Bayes estimation of the priors. These estimated priors are used to construct denoisers within an approximate message passing (AMP) framework, yielding denoised low-dimensional features that borrow strength across related modalities while preserving modality-specific signal. The resulting embeddings are used for downstream supervised prediction. We evaluate the framework through simulations under varying dependence structures and signal regimes, comparing against several benchmark methods, and demonstrate its practical utility on two multimodal datasets, namely a trimodal TEA-seq dataset (Swanson et al., 2021) and TCGA-BRCA dataset (Goldman et al., 2020). In the first example, we predict the expression level of a T-cell differentiation marker protein and in the second case we analyze patient survival prediction based on multimodal information. Our method competes with or outperforms the state-of-the-art techniques in both prediction problems, demonstrating its versatility across diverse supervised learning tasks.