# Bayesian Semi-structured Subspace Inference

arXiv: https://arxiv.org/abs/2401.12950 · HTML: https://mlsanity.com/paper/2401.12950/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Subspace Inference for Bayesian Deep Learning](https://mlsanity.com/paper/1907.07504/) (2019)
- [Learning Neural Network Subspaces](https://mlsanity.com/paper/2102.10472/) (2021)
- [Semi-Structured Distributional Regression -- Extending Structured Additive Models by Arbitrary Deep Neural Networks and Data Modalities](https://mlsanity.com/paper/2002.05777/) (2020)
- [Deep Conditional Transformation Models](https://mlsanity.com/paper/2010.07860/) (2020)
- [Deep transformation models: Tackling complex regression problems with neural network based transformation models](https://mlsanity.com/paper/2004.00464/) (2020)
- [A Wide and Deep Neural Network for Survival Analysis from Anatomical Shape and Tabular Clinical Data](https://mlsanity.com/paper/1909.03890/) (2019)
- [DeepPAMM: Deep Piecewise Exponential Additive Mixed Models for Complex Hazard Structures in Survival Analysis](https://mlsanity.com/paper/2202.07423/) (2022)
- [A New PHO-rmula for Improved Performance of Semi-Structured Networks](https://mlsanity.com/paper/2306.00522/) (2023)
- [Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks](https://mlsanity.com/paper/2309.03061/) (2023)
- [Deep interpretable ensembles](https://mlsanity.com/paper/2205.12729/) (2022)
- [Bernstein Flows for Flexible Posteriors in Variational Bayes](https://mlsanity.com/paper/2202.05650/) (2022)
- [Deep Bayesian regression models](https://mlsanity.com/paper/1806.02160/) (2018)

## Built on by (most cited first, 3 of 3)

- [Paths and Ambient Spaces in Neural Loss Landscapes](https://mlsanity.com/paper/2503.03382/) (2025)
- [Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks](https://mlsanity.com/paper/2502.02345/) (2025)
- [Compressed Active Subspaces for Scalable Bayesian Inference](https://mlsanity.com/paper/2609.19539/) (2026)

## Abstract (arXiv)

Semi-structured regression models enable the joint modeling of interpretable structured and complex unstructured feature effects. The structured model part is inspired by statistical models and can be used to infer the input-output relationship for features of particular importance. The complex unstructured part defines an arbitrary deep neural network and thereby provides enough flexibility to achieve competitive prediction performance. While these models can also account for aleatoric uncertainty, there is still a lack of work on accounting for epistemic uncertainty. In this paper, we address this problem by presenting a Bayesian approximation for semi-structured regression models using subspace inference. To this end, we extend subspace inference for joint posterior sampling from a full parameter space for structured effects and a subspace for unstructured effects. Apart from this hybrid sampling scheme, our method allows for tunable complexity of the subspace and can capture multiple minima in the loss landscape. Numerical experiments validate our approach's efficacy in recovering structured effect parameter posteriors in semi-structured models and approaching the full-space posterior distribution of MCMC for increasing subspace dimension. Further, our approach exhibits competitive predictive performance across simulated and real-world datasets.