# Dimension reduction via score ratio matching

arXiv: https://arxiv.org/abs/2410.19990 · HTML: https://mlsanity.com/paper/2410.19990/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [An Introduction to Deep Generative Modeling](https://mlsanity.com/paper/2103.05180/) (2021)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)
- [Subspace Diffusion Generative Models](https://mlsanity.com/paper/2205.01490/) (2022)
- [Mining gold from implicit models to improve likelihood-free inference](https://mlsanity.com/paper/1805.12244/) (2018)
- [Dimension-independent likelihood-informed MCMC](https://mlsanity.com/paper/1411.3688/) (2014)
- [Projected Stein Variational Gradient Descent](https://mlsanity.com/paper/2002.03469/) (2020)
- [Likelihood-informed dimension reduction for nonlinear inverse problems](https://mlsanity.com/paper/1403.4680/) (2014)
- [Certified dimension reduction in nonlinear Bayesian inverse problems](https://mlsanity.com/paper/1807.03712/) (2018)
- [Optimal low-rank approximations of Bayesian linear inverse problems](https://mlsanity.com/paper/1407.3463/) (2014)
- [Score Matched Neural Exponential Families for Likelihood-Free Inference](https://mlsanity.com/paper/2012.10903/) (2020)
- [Variational Hamiltonian Monte Carlo via Score Matching](https://mlsanity.com/paper/1602.02219/) (2016)
- [Scalable conditional deep inverse Rosenblatt transports using tensor-trains and gradient-based dimension reduction](https://mlsanity.com/paper/2106.04170/) (2021)
- [Data-Free Likelihood-Informed Dimension Reduction of Bayesian Inverse Problems](https://mlsanity.com/paper/2102.13245/) (2021)

## Built on by (most cited first, 2 of 2)

- [Copula Active Subspaces I: A Score-Covariance Method for Reduced-Order Non-Gaussian Density Estimation](https://mlsanity.com/paper/2609.36142/) (2026)
- [Copula Active Subspaces II: Error Decomposition, A Posteriori Estimation, and Sharpness of the Bounds](https://mlsanity.com/paper/2610.14498/) (2026)

## Abstract (arXiv)

Gradient-based dimension reduction decreases the cost of Bayesian inference and probabilistic modeling by identifying maximally informative (and informed) low-dimensional projections of the data and parameters, allowing high-dimensional problems to be reformulated as cheaper low-dimensional problems. A broad family of such techniques identify these projections and provide error bounds on the resulting posterior approximations, via eigendecompositions of certain diagnostic matrices. Yet these matrices require gradients or even Hessians of the log-likelihood, excluding the purely data-driven setting and many problems of simulation-based inference. We propose a framework, derived from score-matching, to extend gradient-based dimension reduction to problems where gradients are unavailable. Specifically, we formulate an objective function to directly learn the score ratio function needed to compute the diagnostic matrices, propose a tailored parameterization for the score ratio network, and introduce regularization methods that capitalize on the hypothesized low-dimensional structure. We also introduce a novel algorithm to iteratively identify the low-dimensional reduced basis vectors more accurately with limited data based on eigenvalue deflation methods. We show that our approach outperforms standard score-matching for problems with low-dimensional structure, and demonstrate its effectiveness for PDE-constrained Bayesian inverse problems and conditional generative modeling.