# Review and Prospect of Algebraic Research in Equivalent Framework between Statistical Mechanics and Machine Learning Theory

arXiv: https://arxiv.org/abs/2406.10234 · HTML: https://mlsanity.com/paper/2406.10234/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 97 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 14 of 14)

- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Foundational Challenges in Assuring Alignment and Safety of Large Language Models](https://mlsanity.com/paper/2404.09932/) (2024)
- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Understanding predictive information criteria for Bayesian models](https://mlsanity.com/paper/1307.5928/) (2013)
- [Deep Learning is Singular, and That's Good](https://mlsanity.com/paper/2010.11560/) (2020)
- [A Bayesian information criterion for singular models](https://mlsanity.com/paper/1309.0911/) (2013)
- [The Exact Asymptotic Form of Bayesian Generalization Error in Latent Dirichlet Allocation](https://mlsanity.com/paper/2008.01304/) (2020)
- [Using Degeneracy in the Loss Landscape for Mechanistic Interpretability](https://mlsanity.com/paper/2405.10927/) (2024)
- [Upper Bound of Bayesian Generalization Error in Non-negative Matrix Factorization](https://mlsanity.com/paper/1612.04112/) (2016)
- [Asymptotic Behavior of Bayesian Generalization Error in Multinomial Mixtures](https://mlsanity.com/paper/2203.06884/) (2022)
- [Mathematical Theory of Bayesian Statistics for Unknown Information Source](https://mlsanity.com/paper/2206.05630/) (2022)
- [Free energy of Bayesian Convolutional Neural Network with Skip Connection](https://mlsanity.com/paper/2307.01417/) (2023)
- [Posterior Covariance Information Criterion for Weighted Inference](https://mlsanity.com/paper/2106.13694/) (2021)
- [A generalization gap estimation for overparameterized models via the Langevin functional variance](https://mlsanity.com/paper/2112.03660/) (2021)

## Built on by (most cited first, 2 of 2)

- [Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descent](https://mlsanity.com/paper/2503.22478/) (2025)
- [Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks](https://mlsanity.com/paper/2608.06597/) (2026)

## Abstract (arXiv)

Mathematical equivalence between statistical mechanics and machine learning theory has been known since the 20th century, and research based on this equivalence has provided novel methodologies in both theoretical physics and statistical learning theory. It is well known that algebraic approaches in statistical mechanics such as operator algebra enable us to analyze phase transition phenomena mathematically. In this paper, we review and prospect algebraic research in machine learning theory for theoretical physicists who are interested in artificial intelligence. If a learning machine has a hierarchical structure or latent variables, then the random Hamiltonian cannot be expressed by any quadratic perturbation because it has singularities. To study an equilibrium state defined by such a singular random Hamiltonian, algebraic approaches are necessary to derive the asymptotic form of the free energy and the generalization error. We also introduce the most recent advance: the theoretical foundation for the alignment of artificial intelligence is now being constructed based on algebraic learning theory. This paper is devoted to the memory of Professor Huzihiro Araki who is a pioneering founder of algebraic research in both statistical mechanics and quantum field theory.