# An entropy-optimal path to humble AI

arXiv: https://arxiv.org/abs/2506.17940 · HTML: https://mlsanity.com/paper/2506.17940/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 47 references that are in the map.
- First posted on arXiv 22 Jun 2025.

## Builds on (most cited first, 5 of 5)

- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity](https://mlsanity.com/paper/2506.06941/) (2025)
- [From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference](https://mlsanity.com/paper/2310.03003/) (2023)
- [The rising costs of training frontier AI models](https://mlsanity.com/paper/2405.21015/) (2024)
- [On existence, uniqueness and scalability of adversarial robustness measures for AI classifiers](https://mlsanity.com/paper/2310.14421/) (2023)

## Built on by (most cited first, 2 of 2)

- [Distillation and Interpretability of Ensemble Forecasts of ENSO Phase using Entropic Learning](https://mlsanity.com/paper/2602.16857/) (2026)
- [On a joint simultaneous learning of relevant feature subsets and subspaces in regression-like problems](https://mlsanity.com/paper/2607.28080/) (2026)

## Abstract (arXiv)

Progress of AI has led to very successful, but by no means humble models and tools, especially regarding (i) the huge and further exploding costs and resources they demand, and (ii) the over-confidence of these tools with the answers they provide. Here we introduce a novel mathematical framework for a non-equilibrium entropy-optimizing reformulation of Boltzmann machines based on the exact law of total probability and the exact convex polytope representations. We show that it results in the highly-performant, but much cheaper, gradient-descent-free learning framework with mathematically-justified existence and uniqueness criteria, and cheaply-computable confidence/reliability measures for both the model inputs and the outputs. Comparisons to state-of-the-art AI tools in terms of performance, cost and the model descriptor lengths on a broad set of synthetic and real-world problems with varying complexity reveal that the proposed method results in more performant and slim models, with the descriptor lengths being very close to the intrinsic complexity scaling bounds for the underlying problems. Applying this framework to historical climate data results in models with systematically higher prediction skills for the onsets of important La Ni~na and El Ni~no climate phenomena, requiring just few years of climate data for training - a small fraction of what is necessary for contemporary climate prediction tools.