# Empirical Bayes Estimation for Lasso-Type Regularizers: Analysis of Automatic Relevance Determination

arXiv: https://arxiv.org/abs/2501.11280 · HTML: https://mlsanity.com/paper/2501.11280/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 21 references that are in the map.
- First posted on arXiv 20 Jan 2025.

## Builds on (most cited first, 2 of 2)

- [Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration](https://mlsanity.com/paper/1902.10375/) (2019)
- [Empirical Bayes Method for Boltzmann Machines](https://mlsanity.com/paper/1906.06002/) (2019)

## Abstract (arXiv)

This paper focuses on linear regression models with non-conjugate sparsity-inducing regularizers such as lasso and group lasso. Although the empirical Bayes approach enables us to estimate the regularization parameter, little is known on the properties of the estimators. In particular, many aspects regarding the specific conditions under which the mechanism of automatic relevance determination (ARD) occurs remain unexplained. In this paper, we derive the empirical Bayes estimators for the group lasso regularized linear regression models with limited parameters. It is shown that the estimators diverge under a specific condition, giving rise to the ARD mechanism. We also prove that empirical Bayes methods can produce the ARD mechanism in general regularized linear regression models and clarify the conditions under which models such as ridge, lasso, and group lasso can do so.