# Overfitted high-dimensional matrix factorizations via adaptive spectral shrinkage

arXiv: https://arxiv.org/abs/2606.19540 · HTML: https://mlsanity.com/paper/2606.19540/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 23 of 23)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [On the Non-asymptotic and Sharp Lower Tail Bounds of Random Variables](https://mlsanity.com/paper/1810.09006/) (2018)
- [Bayesian cumulative shrinkage for infinite factorizations](https://mlsanity.com/paper/1902.04349/) (2019)
- [Structured Latent Factor Analysis for Large-scale Data: Identifiability, Estimability, and Their Implications](https://mlsanity.com/paper/1712.08966/) (2017)
- [Bridging factor and sparse models](https://mlsanity.com/paper/2102.11341/) (2021)
- [Determining the Number of Factors in High-dimensional Generalized Latent Factor Models](https://mlsanity.com/paper/2010.02326/) (2020)
- [Compressive Bayesian non-negative matrix factorization for mutational signatures analysis](https://mlsanity.com/paper/2404.10974/) (2024)
- [Spectral decomposition-assisted multi-study factor analysis](https://mlsanity.com/paper/2502.14600/) (2025)
- [A note on the multiplicative gamma process](https://mlsanity.com/paper/1610.03408/) (2016)
- [Blessing of dimension in Bayesian inference on covariance matrices](https://mlsanity.com/paper/2404.03805/) (2024)
- [A Latent Variable Approach to Learning High-dimensional Multivariate longitudinal Data](https://mlsanity.com/paper/2405.15053/) (2024)
- [Bayesian Factor-adjusted Sparse Regression](https://mlsanity.com/paper/1903.09741/) (2019)
- [Semiparametric Functional Factor Models with Bayesian Rank Selection](https://mlsanity.com/paper/2108.02151/) (2021)
- [Factor pre-training in Bayesian multivariate logistic models](https://mlsanity.com/paper/2409.17441/) (2024)
- [bayesNMF: Fast Bayesian Poisson NMF with Automatically Learned Rank Applied to Mutational Signatures](https://mlsanity.com/paper/2502.18674/) (2025)
- [Consistent estimation of high-dimensional factor models when the factor number is over-estimated](https://mlsanity.com/paper/1811.00306/) (2018)
- [Simpler Proofs for Approximate Factor Models of Large Dimensions](https://mlsanity.com/paper/2008.00254/) (2020)
- [Statistical inference of random graphs with a surrogate likelihood function](https://mlsanity.com/paper/2207.01702/) (2022)
- [Multivariate Causal Effects: a Bayesian Causal Regression Factor Model](https://mlsanity.com/paper/2504.03480/) (2025)
- [SANVI: A Fast Spectral-Assisted Network Variational Inference Method with an Extended Surrogate Likelihood Function](https://mlsanity.com/paper/2509.00562/) (2025)
- [Inference on covariance structure in high-dimensional multi-view data](https://mlsanity.com/paper/2509.02772/) (2025)

## Built on by (most cited first, 1 of 1)

- [Empirical-Bayes spectral partial pooling across related tasks](https://mlsanity.com/paper/2610.07284/) (2026)

## Abstract (arXiv)

Factor models are popular approaches for analyzing high-dimensional data to extract low-rank signals and estimate covariances. They decompose the covariance matrix as the sum of low-rank and diagonal components. A key issue is how to choose the latent dimension k, which is particularly challenging when the factor model only holds approximately and in low signal-to-noise scenarios. Bayesian overfitted factor models specify an upper bound on k and rely on structured shrinkage priors to effectively remove extra components. Such approaches are popular and effective, but computationally expensive. We propose a much faster textttEigenBayes approach that provides valid uncertainty quantification, based on spectral estimation of latent factors and adaptive empirical Bayes calibration of key hyperparameters. The resulting posterior distribution factorizes across outcomes and is analytically tractable, bypassing Markov chain Monte Carlo. We show that textttEigenBayes adapts to the signal-to-noise ratio of each outcome and latent dimension, while shrinking superfluous latent components to zero. We establish favorable asymptotic properties and demonstrate strong empirical performance in numerical experiments and a genomics application, where EigenBayes outperforms state-of-the-art alternatives.