# Bayes and Biased Estimators Without Hyper-parameter Estimation: Comparable Performance to the Empirical-Bayes-Based Regularized Estimator

arXiv: https://arxiv.org/abs/2503.11854 · HTML: https://mlsanity.com/paper/2503.11854/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 6 of 6)

- [On kernel design for regularized LTI system identification](https://mlsanity.com/paper/1612.03542/) (2016)
- [On Asymptotic Properties of Hyperparameter Estimators for Kernel-based Regularization Methods](https://mlsanity.com/paper/1707.00407/) (2017)
- [A new kernel-based approach for overparameterized Hammerstein system identification](https://mlsanity.com/paper/1504.08190/) (2015)
- [Learning sparse linear dynamic networks in a hyper-parameter free setting](https://mlsanity.com/paper/1911.11553/) (2019)
- [A second-order generalization of TC and DC kernels](https://mlsanity.com/paper/2109.09562/) (2021)
- [On Kernel Design for Regularized Non-Causal System Identification](https://mlsanity.com/paper/2307.13999/) (2023)

## Built on by (most cited first, 1 of 1)

- [XMSE-Aware Adaptive Empirical Bayes Estimation](https://mlsanity.com/paper/2606.26975/) (2026)

## Abstract (arXiv)

Regularized system identification has become a significant complement to more classical system identification. It has been numerically shown that kernel-based regularized estimators often perform better than the maximum likelihood estimator in terms of minimizing mean squared error (MSE). However, regularized estimators often require hyper-parameter estimation. This paper focuses on ridge regression and the regularized estimator by employing the empirical Bayes hyper-parameter estimator. We utilize the excess MSE to quantify the MSE difference between the empirical-Bayes-based regularized estimator and the maximum likelihood estimator for large sample sizes. We then exploit the excess MSE expressions to develop both a family of generalized Bayes estimators and a family of closed-form biased estimators. They have the same excess MSE as the empirical-Bayes-based regularized estimator but eliminate the need for hyper-parameter estimation. Moreover, we conduct numerical simulations to show that the performance of these new estimators is comparable to the empirical-Bayes-based regularized estimator, while computationally, they are more efficient.