# Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks

arXiv: https://arxiv.org/abs/2309.03061 · HTML: https://mlsanity.com/paper/2309.03061/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Subspace Inference for Bayesian Deep Learning](https://mlsanity.com/paper/1907.07504/) (2019)
- [Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited](https://mlsanity.com/paper/2003.02139/) (2020)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Likelihood-informed dimension reduction for nonlinear inverse problems](https://mlsanity.com/paper/1403.4680/) (2014)
- [Bayesian Inference with Certifiable Adversarial Robustness](https://mlsanity.com/paper/2102.05289/) (2021)
- [Active Subspace of Neural Networks: Structural Analysis and Universal Attacks](https://mlsanity.com/paper/1910.13025/) (2019)
- [Deep active subspaces - a scalable method for high-dimensional uncertainty propagation](https://mlsanity.com/paper/1902.10527/) (2019)

## Built on by (most cited first, 9 of 9)

- [C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models](https://mlsanity.com/paper/2505.17773/) (2025)
- [Bayesian Semi-structured Subspace Inference](https://mlsanity.com/paper/2401.12950/) (2024)
- [Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design](https://mlsanity.com/paper/2405.00202/) (2024)
- [Enhancing Generative Molecular Design via Uncertainty-guided Fine-tuning of Variational Autoencoders](https://mlsanity.com/paper/2405.20573/) (2024)
- [Distance-Aware Error for Spline Networks: A Bottom-Up Approach to Uncertainty](https://mlsanity.com/paper/2501.04757/) (2025)
- [Uncertainty-Aware Adaptation of Large Language Models for Protein-Protein Interaction Analysis](https://mlsanity.com/paper/2502.06173/) (2025)
- [Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators](https://mlsanity.com/paper/2507.14652/) (2025)
- [Accurate Large-sample Uncertainty Quantification using Stochastic Gradient Markov Chain Monte Carlo](https://mlsanity.com/paper/2606.00293/) (2026)
- [Compressed Active Subspaces for Scalable Bayesian Inference](https://mlsanity.com/paper/2609.19539/) (2026)

## Abstract (arXiv)

Bayesian inference for neural networks, or Bayesian deep learning, has the potential to provide well-calibrated predictions with quantified uncertainty and robustness. However, the main hurdle for Bayesian deep learning is its computational complexity due to the high dimensionality of the parameter space. In this work, we propose a novel scheme that addresses this limitation by constructing a low-dimensional subspace of the neural network parameters-referred to as an active subspace-by identifying the parameter directions that have the most significant influence on the output of the neural network. We demonstrate that the significantly reduced active subspace enables effective and scalable Bayesian inference via either Monte Carlo (MC) sampling methods, otherwise computationally intractable, or variational inference. Empirically, our approach provides reliable predictions with robust uncertainty estimates for various regression tasks.