# The Physics of Learning: From Autoencoders to Truly Autonomous Learning Machines

arXiv: https://arxiv.org/abs/2407.04700 · HTML: https://mlsanity.com/paper/2407.04700/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 7 of 7)

- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Meta-Learning Online Adaptation of Language Models](https://mlsanity.com/paper/2305.15076/) (2023)
- [Evolutionary Reinforcement Learning: A Survey](https://mlsanity.com/paper/2303.04150/) (2023)
- [On Principles of Emergent Organization](https://mlsanity.com/paper/2311.13749/) (2023)
- [Can Turing machine be curious about its Turing test results? Three informal lectures on physics of intelligence](https://mlsanity.com/paper/1606.08109/) (2016)
- [Understanding Learning through the Lens of Dynamical Invariants](https://mlsanity.com/paper/2401.10428/) (2024)

## Abstract (arXiv)

The fact that accurately predicted information can serve as an energy source paves the way for new approaches to autonomous learning. The energy derived from a sequence of successful predictions can be recycled as an immediate incentive and resource, driving the enhancement of predictive capabilities in AI agents. We propose that, through a series of straightforward meta-architectural adjustments, any unsupervised learning apparatus could achieve complete independence from external energy sources, evolving into a self-sustaining physical system with a strong intrinsic 'drive' for continual learning. This concept, while still purely theoretical, is exemplified through the autoencoder, a quintessential model for unsupervised efficient coding. We use this model to demonstrate how progressive paradigm shifts can profoundly alter our comprehension of learning and intelligence. By reconceptualizing learning as an energy-seeking process, we highlight the potential for achieving true autonomy in learning systems, thereby bridging the gap between algorithmic concepts and physical models of intelligence.