# Emergence of hybrid computational dynamics through reinforcement learning

arXiv: https://arxiv.org/abs/2510.11162 · HTML: https://mlsanity.com/paper/2510.11162/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 6 of 6)

- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Attractor and integrator networks in the brain](https://mlsanity.com/paper/2112.03978/) (2021)
- [Deep Reinforcement Learning and its Neuroscientific Implications](https://mlsanity.com/paper/2007.03750/) (2020)
- [Artificial neural networks for neuroscientists: A primer](https://mlsanity.com/paper/2006.01001/) (2020)
- [Neural Network Compression for Reinforcement Learning Tasks](https://mlsanity.com/paper/2405.07748/) (2024)
- [Fast gradient-free activation maximization for neurons in spiking neural networks](https://mlsanity.com/paper/2401.10748/) (2023)

## Abstract (arXiv)

Understanding how learning algorithms shape the computational strategies that emerge in neural networks remains a fundamental challenge in machine intelligence. While network architectures receive extensive attention, the role of the learning paradigm itself in determining emergent dynamics remains largely unexplored. Here we demonstrate that reinforcement learning (RL) and supervised learning (SL) drive recurrent neural networks (RNNs) toward fundamentally different computational solutions when trained on identical decision-making tasks. Through systematic dynamical systems analysis, we reveal that RL spontaneously discovers hybrid attractor architectures, combining stable fixed-point attractors for decision maintenance with quasi-periodic attractors for flexible evidence integration. This contrasts sharply with SL, which converges almost exclusively to simpler fixed-point-only solutions. We further show that RL sculpts functionally balanced neural populations through a powerful form of implicit regularization -- a structural signature that enhances robustness and is conspicuously absent in the more heterogeneous solutions found by SL-trained networks. The prevalence of these complex dynamics in RL is controllably modulated by weight initialization and correlates strongly with performance gains, particularly as task complexity increases. Our results establish the learning algorithm as a primary determinant of emergent computation, revealing how reward-based optimization autonomously discovers sophisticated dynamical mechanisms that are less accessible to direct gradient-based optimization. These findings provide both mechanistic insights into neural computation and actionable principles for designing adaptive AI systems.