# Endogenous Regime Switching Driven by Scalar-Irreducible Learning Dynamics

arXiv: https://arxiv.org/abs/2605.04054 · HTML: https://mlsanity.com/paper/2605.04054/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [The large learning rate phase of deep learning: the catapult mechanism](https://mlsanity.com/paper/2003.02218/) (2020)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [Adding Gradient Noise Improves Learning for Very Deep Networks](https://mlsanity.com/paper/1511.06807/) (2015)
- [Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods](https://mlsanity.com/paper/2404.00282/) (2024)
- [Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks](https://mlsanity.com/paper/1710.11029/) (2017)
- [EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training](https://mlsanity.com/paper/2405.08768/) (2024)
- [Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization](https://mlsanity.com/paper/2206.05658/) (2022)
- [Learning Rate Curriculum](https://mlsanity.com/paper/2205.09180/) (2022)
- [Gradient descent provably escapes saddle points in the training of shallow ReLU networks](https://mlsanity.com/paper/2208.02083/) (2022)
- [LLM-Driven Intrinsic Motivation for Sparse Reward Reinforcement Learning](https://mlsanity.com/paper/2508.18420/) (2025)

## Abstract (arXiv)

Achieving endogenous regime switching is crucial for the emergence of autonomous intelligence, yet remains a central challenge for existing machine learning frameworks, where such transitions are typically externally imposed. In this work, we introduce a classification that distinguishes scalar-reducible dynamics, which can be expressed as gradient flows driven by a scalar objective, from scalar-irreducible dynamics that cannot be reduced to such a form. While most existing machine learning systems operate within the scalar-reducible class, we demonstrate that scalar-irreducible dynamics naturally enable internally generated regime switching through feedback between fast dynamical variables and slow structural adaptation. Using a minimal dynamical model, we illustrate how this mechanism produces sustained endogenous regime transitions without external scheduling. Our results suggest a new dynamical paradigm for regime exploration and provide a potential route toward autonomous learning systems whose adaptive behavior is organized internally rather than externally prescribed.