mlsanityData to 7 Oct 2026

Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions

arXiv 2407.03522 ↗First posted 3 Jul 2024stat.ML cond-mat.dis-nn cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

This work explores multi-modal inference in a high-dimensional simplified model, analytically quantifying the performance gain of multi-modal inference over that of analyzing modalities in isolation. We present the Bayes-optimal performance and recovery thresholds in a model where the objective is to recover the latent structures from two noisy data matrices with correlated spikes. The paper derives the approximate message passing (AMP) algorithm for this model and characterizes its performance in the high-dimensional limit via the associated state evolution. The analysis holds for a broad range of priors and noise channels, which can differ across modalities. The linearization of AMP is compared numerically to the widely used partial least squares (PLS) and canonical correlation analysis (CCA) methods, which are both observed to suffer from a sub-optimal recovery threshold.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 61 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

24 of 61 references.

References · 24

  1. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  2. 2023Multimodal Deep Learningcited by 257
  3. 2023A Cookbook of Self-Supervised Learningcited by 248
  4. 2018Contextual Stochastic Block Modelscited by 134
  5. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
  6. 2013Spectral redemption: clustering sparse networkscited by 54
  7. 2021A unifying tutorial on Approximate Message Passingcited by 49
  8. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
  9. 2020The estimation error of general first order methodscited by 33
  10. 2018Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inferencecited by 24
  11. 2015MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channelcited by 22
  12. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
12 more
  1. 2020Empirical Bayes PCA in high dimensionscited by 11
  2. 2020Information-Theoretic Limits for the Matrix Tensor Productcited by 9
  3. 2018Detection limits in the high-dimensional spiked rectangular modelcited by 8
  4. 2024Fundamental limits of community detection from multi-view data: multi-layer, dynamic and partially labeled block modelscited by 7
  5. 2023Approximate Message Passing for the Matrix Tensor Product Modelcited by 6
  6. 2023Simultaneous Dimensionality Reduction: A Data Efficient Approach for Multimodal Representations Learningcited by 6
  7. 2022One-Way Matching of Datasets with Low Rank Signalscited by 5
  8. 2024Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methodscited by 5
  9. 2018Statistical mechanics of low-rank tensor decompositioncited by 4
  10. 2023Neural-prior stochastic block modelcited by 4
  11. 2024Spectral Phase Transition and Optimal PCA in Block-Structured Spiked modelscited by 4
  12. 2023Bayes optimal learning in high-dimensional linear regression with network side informationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Modelcited by 5
  2. 2025Spectral Thresholds in Correlated Spiked Models and Fundamental Limits of Partial Least Squarescited by 4
  3. 2026Multi-layer Cross-attention is Provably Optimal for Multi-modal In-context Learningcited by 2
  4. 2026DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passingcited by 0