# Sparse Methods for Automatic Relevance Determination

arXiv: https://arxiv.org/abs/2005.08741 · HTML: https://mlsanity.com/paper/2005.08741/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 8 of 8)

- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization](https://mlsanity.com/paper/2206.00176/) (2022)
- [Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case](https://mlsanity.com/paper/2110.00546/) (2021)
- [A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertainty](https://mlsanity.com/paper/2403.08901/) (2024)
- [A Sparse Bayesian Learning for Diagnosis of Nonstationary and Spatially Correlated Faults with Application to Multistation Assembly Systems](https://mlsanity.com/paper/2310.16058/) (2023)
- [Automatic dimensionality reduction of Twin-in-the-Loop Observers](https://mlsanity.com/paper/2401.10945/) (2024)
- [Sparse Kalman Identification for Partially Observable Systems via Adaptive Bayesian Learning](https://mlsanity.com/paper/2511.18051/) (2025)
- [Joint Model and Data Sparsification via the Marginal Likelihood](https://mlsanity.com/paper/2605.29908/) (2026)

## Abstract (arXiv)

This work considers methods for imposing sparsity in Bayesian regression with applications in nonlinear system identification. We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems. In the case of orthogonal covariates, we analytically demonstrate favorable performance with regards to learning a small set of active terms in a linear system with a sparse solution. Several example problems are presented to compare the set of proposed methods in terms of advantages and limitations to ARD in bases with hundreds of elements. The aim of this paper is to analyze and understand the assumptions that lead to several algorithms and to provide theoretical and empirical results so that the reader may gain insight and make more informed choices regarding sparse Bayesian regression.