# Learning under Singularity: An Information Criterion improving WBIC and sBIC

arXiv: https://arxiv.org/abs/2402.12762 · HTML: https://mlsanity.com/paper/2402.12762/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 5 of 5)

- [A Bayesian information criterion for singular models](https://mlsanity.com/paper/1309.0911/) (2013)
- [Marginal likelihood and model selection for Gaussian latent tree and forest models](https://mlsanity.com/paper/1412.8285/) (2014)
- [Estimating Real Log Canonical Thresholds](https://mlsanity.com/paper/1906.01341/) (2019)
- [Mathematical Theory of Bayesian Statistics for Unknown Information Source](https://mlsanity.com/paper/2206.05630/) (2022)
- [On the overestimation of widely applicable Bayesian information criterion](https://mlsanity.com/paper/1908.10572/) (2019)

## Built on by (most cited first, 1 of 1)

- [Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Models](https://mlsanity.com/paper/2608.20183/) (2026)

## Abstract (arXiv)

We introduce a novel Information Criterion (IC), termed Learning under Singularity (LS), designed to enhance the functionality of the Widely Applicable Bayes Information Criterion (WBIC) and the Singular Bayesian Information Criterion (sBIC). LS is effective without regularity constraints and demonstrates stability. Watanabe defined a statistical model or a learning machine as regular if the mapping from a parameter to a probability distribution is one-to-one and its Fisher information matrix is positive definite. In contrast, models not meeting these conditions are termed singular. Over the past decade, several information criteria for singular cases have been proposed, including WBIC and sBIC. WBIC is applicable in non-regular scenarios but faces challenges with large sample sizes and redundant estimation of known learning coefficients. Conversely, sBIC is limited in its broader application due to its dependence on maximum likelihood estimates. LS addresses these limitations by enhancing the utility of both WBIC and sBIC. It incorporates the empirical loss from the Widely Applicable Information Criterion (WAIC) to represent the goodness of fit to the statistical model, along with a penalty term similar to that of sBIC. This approach offers a flexible and robust method for model selection, free from regularity constraints.