# Information-Geometric Decomposition of Generalization Error in Unsupervised Learning

arXiv: https://arxiv.org/abs/2604.12340 · HTML: https://mlsanity.com/paper/2604.12340/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 23 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 1 of 1)

- [Tradeoff of generalization error in unsupervised learning](https://mlsanity.com/paper/2303.05718/) (2023)

## Abstract (arXiv)

We decompose the Kullback--Leibler generalization error (GE) -- the expected KL divergence from the data distribution to the trained model -- of unsupervised learning into three non-negative components: model error, data bias, and variance. The decomposition is exact for any e-flat model class and follows from two identities of information geometry: the generalized Pythagorean theorem and a dual e-mixture variance identity. As an analytically tractable demonstration, we apply the framework to ε-PCA, a regularized principal component analysis in which the empirical covariance is truncated at rank NK and discarded directions are pinned at a fixed noise floor ε. Although rank-constrained ε-PCA is not itself e-flat, it admits a technical reformulation with the same total GE on isotropic Gaussian data, under which each component of the decomposition takes closed form. The optimal rank emerges as the cutoff λcut* = ε -- the model retains exactly those empirical eigenvalues exceeding the noise floor -- with the cutoff reflecting a marginal-rate balance between model-error gain and data-bias cost. A boundary comparison further yields a three-regime phase diagram -- retain-all, interior, and collapse -- separated by the lower Marchenko--Pastur edge and an analytically computable collapse threshold ε*(α), where α is the dimension-to-sample-size ratio. All claims are verified numerically.