# Loss-Complexity Landscape and Model Structure Functions

arXiv: https://arxiv.org/abs/2507.13543 · HTML: https://mlsanity.com/paper/2507.13543/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 21 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Builds on (most cited first, 2 of 2)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [A Modern Take on the Bias-Variance Tradeoff in Neural Networks](https://mlsanity.com/paper/1810.08591/) (2018)

## Abstract (arXiv)

We develop a framework for dualizing the Kolmogorov structure function hx(α), which then allows using computable complexity proxies. We establish a mathematical analogy between information-theoretic constructs and statistical mechanics, introducing a suitable partition function and free energy functional. We explicitly prove the Legendre-Fenchel duality between the structure function and free energy, showing detailed balance of the Metropolis kernel, and interpret acceptance probabilities as information-theoretic scattering amplitudes. A susceptibility-like variance of model complexity is shown to peak precisely at loss-complexity trade-offs interpreted as phase transitions. Practical experiments with linear and tree-based regression models verify these theoretical predictions, explicitly demonstrating the interplay between the model complexity, generalization, and overfitting threshold.