# Optimal low-rank approximations of Bayesian linear inverse problems

arXiv: https://arxiv.org/abs/1407.3463 · HTML: https://mlsanity.com/paper/1407.3463/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2014.

## Built on by (most cited first, 22 of 22)

- [Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problems](https://mlsanity.com/paper/2406.02462/) (2024)
- [A low-rank ensemble Kalman filter for elliptic observations](https://mlsanity.com/paper/2203.05120/) (2022)
- [Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference](https://mlsanity.com/paper/2105.05489/) (2021)
- [Expected information gain estimation via density approximations: Sample allocation and dimension reduction](https://mlsanity.com/paper/2411.08390/) (2024)
- [Sharp detection of low-dimensional structure in probability measures via dimensional logarithmic Sobolev inequalities](https://mlsanity.com/paper/2406.13036/) (2024)
- [Accelerating Non-Conjugate Gaussian Processes By Trading Off Computation For Uncertainty](https://mlsanity.com/paper/2310.20285/) (2023)
- [Dimension reduction via score ratio matching](https://mlsanity.com/paper/2410.19990/) (2024)
- [A Data-Adaptive Prior for Bayesian Learning of Kernels in Operators](https://mlsanity.com/paper/2212.14163/) (2022)
- [Meta-Continual Learning of Neural Fields](https://mlsanity.com/paper/2504.05806/) (2025)
- [A Mathematical Perspective On Contrastive Learning](https://mlsanity.com/paper/2505.24134/) (2025)
- [Latent-IMH: Efficient Bayesian Inference for Inverse Problems with Approximate Operators](https://mlsanity.com/paper/2601.20888/) (2026)
- [Stochastic Learning Approach to Binary Optimization for Optimal Design of Experiments](https://mlsanity.com/paper/2101.05958/) (2021)
- [Posterior sampling with CNN-based, Plug-and-Play regularization with applications to Post-Stack Seismic Inversion](https://mlsanity.com/paper/2212.14595/) (2022)
- [How accurate is mechanobiology? A statistical test of cell force](https://mlsanity.com/paper/2412.18406/) (2024)
- [Optimal Linear Baseline Models for Scientific Machine Learning](https://mlsanity.com/paper/2508.05831/) (2025)
- [Proximal-IMH: Proximal Posterior Proposals for Independent Metropolis-Hastings with Approximate Operators](https://mlsanity.com/paper/2602.21426/) (2026)
- [SPDE Methods for Nonparametric Bayesian Posterior Contraction and Laplace Approximation](https://mlsanity.com/paper/2603.22468/) (2026)
- [Priors learned from legacy reconstructions inherit undetectable overconfidence](https://mlsanity.com/paper/2607.21721/) (2026)
- [Fast rates in Bayesian online learning with approximate posteriors](https://mlsanity.com/paper/2608.25706/) (2026)
- [Diffusion Based Unpaired Data Learning for Inverse Problems](https://mlsanity.com/paper/2609.01370/) (2026)
- [When Noise Estimation Hides Basis Misspecification in Repeated Bayesian Inverse Problems](https://mlsanity.com/paper/2609.37762/) (2026)
- [Measuring Effective Data Resolution in Guided Diffusion Posteriors](https://mlsanity.com/paper/2610.04422/) (2026)

## Abstract (arXiv)

In the Bayesian approach to inverse problems, data are often informative, relative to the prior, only on a low-dimensional subspace of the parameter space. Significant computational savings can be achieved by using this subspace to characterize and approximate the posterior distribution of the parameters. We first investigate approximation of the posterior covariance matrix as a low-rank update of the prior covariance matrix. We prove optimality of a particular update, based on the leading eigendirections of the matrix pencil defined by the Hessian of the negative log-likelihood and the prior precision, for a broad class of loss functions. This class includes the F"orstner metric for symmetric positive definite matrices, as well as the Kullback-Leibler divergence and the Hellinger distance between the associated distributions. We also propose two fast approximations of the posterior mean and prove their optimality with respect to a weighted Bayes risk under squared-error loss. These approximations are deployed in an offline-online manner, where a more costly but data-independent offline calculation is followed by fast online evaluations. As a result, these approximations are particularly useful when repeated posterior mean evaluations are required for multiple data sets. We demonstrate our theoretical results with several numerical examples, including high-dimensional X-ray tomography and an inverse heat conduction problem. In both of these examples, the intrinsic low-dimensional structure of the inference problem can be exploited while producing results that are essentially indistinguishable from solutions computed in the full space.