# Thermodynamic limit in learning period three

arXiv: https://arxiv.org/abs/2405.08825 · HTML: https://mlsanity.com/paper/2405.08825/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 73 references that are in the map.
- First posted on arXiv 12 May 2024.

## Builds on (most cited first, 23 of 23)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [On Exact Computation with an Infinitely Wide Neural Net](https://mlsanity.com/paper/1904.11955/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [High-dimensional dynamics of generalization error in neural networks](https://mlsanity.com/paper/1710.03667/) (2017)
- [Just Interpolate: Kernel "Ridgeless" Regression Can Generalize](https://mlsanity.com/paper/1808.00387/) (2018)
- [A Random Matrix Approach to Neural Networks](https://mlsanity.com/paper/1702.05419/) (2017)
- [A Continuous-Time View of Early Stopping for Least Squares](https://mlsanity.com/paper/1810.10082/) (2018)
- [Physical reservoir computing -- An introductory perspective](https://mlsanity.com/paper/2005.00992/) (2020)
- [Machine learning prediction of critical transition and system collapse](https://mlsanity.com/paper/2012.01545/) (2020)
- [Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions](https://mlsanity.com/paper/1905.06076/) (2019)
- [Anticipating synchronization with machine learning](https://mlsanity.com/paper/2103.13358/) (2021)
- [Multifunctionality in a Reservoir Computer](https://mlsanity.com/paper/2008.06348/) (2020)
- [Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing](https://mlsanity.com/paper/2108.04074/) (2021)
- [Theory of neuromorphic computing by waves: machine learning by rogue waves, dispersive shocks, and solitons](https://mlsanity.com/paper/1912.07044/) (2019)
- [Fast Neural Kernel Embeddings for General Activations](https://mlsanity.com/paper/2209.04121/) (2022)
- [Learning strange attractors with reservoir systems](https://mlsanity.com/paper/2108.05024/) (2021)
- [Reservoir Computing meets Recurrent Kernels and Structured Transforms](https://mlsanity.com/paper/2006.07310/) (2020)
- [The Positivity of the Neural Tangent Kernel](https://mlsanity.com/paper/2404.12928/) (2024)
- [Seeing double with a multifunctional reservoir computer](https://mlsanity.com/paper/2305.05799/) (2023)
- [Kernel Methods and Multi-layer Perceptrons Learn Linear Models in High Dimensions](https://mlsanity.com/paper/2201.08082/) (2022)
- [Embedding bifurcations into pneumatic artificial muscle](https://mlsanity.com/paper/2305.03994/) (2023)
- [Asymptotic Stability in Reservoir Computing](https://mlsanity.com/paper/2206.03854/) (2022)

## Built on by (most cited first, 1 of 1)

- [Multifunctional physical reservoir computing in soft tensegrity robots](https://mlsanity.com/paper/2507.21496/) (2025)

## Abstract (arXiv)

A continuous one-dimensional map with period three includes all periods. This raises the following question: Can we obtain any types of periodic orbits solely by learning three data points? In this paper, we report the answer to be yes. Considering a random neural network in its thermodynamic limit, we first show that almost all learned periods are unstable, and each network has its own characteristic attractors (which can even be untrained ones). The latently acquired dynamics, which are unstable within the trained network, serve as a foundation for the diversity of characteristic attractors and may even lead to the emergence of attractors of all periods after learning. When the neural network interpolation is quadratic, a universal post-learning bifurcation scenario appears, which is consistent with a topological conjugacy between the trained network and the classical logistic map. In addition to universality, we explore specific properties of certain networks, including the singular behavior of the scale of weight at the infinite limit, the finite-size effects, and the symmetry in learning period three.