# Unexpected Benefits of Self-Modeling in Neural Systems

arXiv: https://arxiv.org/abs/2407.10188 · HTML: https://mlsanity.com/paper/2407.10188/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 6 references that are in the map.
- First posted on arXiv 14 Jul 2024.

## Built on by (most cited first, 5 of 5)

- [Heuristically Adaptive Diffusion-Model Evolutionary Strategy](https://mlsanity.com/paper/2411.13420/) (2024)
- [Towards Safe and Honest AI Agents with Neural Self-Other Overlap](https://mlsanity.com/paper/2412.16325/) (2024)
- [Testing Components of the Attention Schema Theory in Artificial Neural Networks](https://mlsanity.com/paper/2411.00983/) (2024)
- [Phase Transitions in Driven Informational Systems: A Two-Field Perspective on Learning Theory and Non-Equilibrium Chemistry](https://mlsanity.com/paper/2605.16325/) (2026)
- [Emergent Language as an Approach to Conscious AI](https://mlsanity.com/paper/2606.06380/) (2026)

## Abstract (arXiv)

Self-models have been a topic of great interest for decades in studies of human cognition and more recently in machine learning. Yet what benefits do self-models confer? Here we show that when artificial networks learn to predict their internal states as an auxiliary task, they change in a fundamental way. To better perform the self-model task, the network learns to make itself simpler, more regularized, more parameter-efficient, and therefore more amenable to being predictively modeled. To test the hypothesis of self-regularizing through self-modeling, we used a range of network architectures performing three classification tasks across two modalities. In all cases, adding self-modeling caused a significant reduction in network complexity. The reduction was observed in two ways. First, the distribution of weights was narrower when self-modeling was present. Second, a measure of network complexity, the real log canonical threshold (RLCT), was smaller when self-modeling was present. Not only were measures of complexity reduced, but the reduction became more pronounced as greater training weight was placed on the auxiliary task of self-modeling. These results strongly support the hypothesis that self-modeling is more than simply a network learning to predict itself. The learning has a restructuring effect, reducing complexity and increasing parameter efficiency. This self-regularization may help explain some of the benefits of self-models reported in recent machine learning literature, as well as the adaptive value of self-models to biological systems. In particular, these findings may shed light on the possible interaction between the ability to model oneself and the ability to be more easily modeled by others in a social or cooperative context.