# Learning dynamical systems with hit-and-run random feature maps

arXiv: https://arxiv.org/abs/2501.06661 · HTML: https://mlsanity.com/paper/2501.06661/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 62 references that are in the map.
- First posted on arXiv 11 Jan 2025.

## Builds on (most cited first, 22 of 22)

- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [Continuous-in-Depth Neural Networks](https://mlsanity.com/paper/2008.02389/) (2020)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/2201.08910/) (2022)
- [Learning continuous models for continuous physics](https://mlsanity.com/paper/2202.08494/) (2022)
- [Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation](https://mlsanity.com/paper/2007.07383/) (2020)
- [Precise Asymptotic Analysis of Deep Random Feature Models](https://mlsanity.com/paper/2302.06210/) (2023)
- [Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations](https://mlsanity.com/paper/2108.03561/) (2021)
- [Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation](https://mlsanity.com/paper/2109.12269/) (2021)
- [Discovery of Dynamics Using Linear Multistep Methods](https://mlsanity.com/paper/1912.12728/) (2019)
- [The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation](https://mlsanity.com/paper/2103.11488/) (2021)
- [Scalable Spatiotemporal Prediction with Bayesian Neural Fields](https://mlsanity.com/paper/2403.07657/) (2024)
- [Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm](https://mlsanity.com/paper/2008.04222/) (2020)
- [Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features](https://mlsanity.com/paper/2407.00584/) (2024)
- [Robust learning of data anomalies with analytically-solvable entropic outlier sparsification](https://mlsanity.com/paper/2112.11768/) (2021)
- [Representation Learning with Deep Extreme Learning Machines for Efficient Image Set Classification](https://mlsanity.com/paper/1503.02445/) (2015)
- [Localized Schr"odinger Bridge Sampler](https://mlsanity.com/paper/2409.07968/) (2024)
- [On the choice of the non-trainable internal weights in random feature maps](https://mlsanity.com/paper/2408.03626/) (2024)

## Built on by (most cited first, 8 of 8)

- [Dynamic Reservoir Computing with Physical Neuromorphic Networks](https://mlsanity.com/paper/2505.16813/) (2025)
- [Learning solution operator of dynamical systems with diffusion maps kernel ridge regression](https://mlsanity.com/paper/2512.17203/) (2025)
- [Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series](https://mlsanity.com/paper/2507.08738/) (2025)
- [Learning Chaotic Dynamics with Neuromorphic Network Dynamics](https://mlsanity.com/paper/2506.10773/) (2025)
- [Machine-Precision Prediction of Low-Dimensional Chaotic Systems from Noise-Free Data](https://mlsanity.com/paper/2507.09652/) (2025)
- [A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series](https://mlsanity.com/paper/2511.06609/) (2025)
- [AugRelNet: Relation-Augmented Dynamics for Structure Discovery and Forecasting from Limited Data](https://mlsanity.com/paper/2606.11251/) (2026)
- [Learning Closure of Dynamical Systems with Kernel Ridge Regression](https://mlsanity.com/paper/2610.02564/) (2026)

## Abstract (arXiv)

We show how random feature maps can be used to forecast dynamical systems with excellent forecasting skill. We consider the tanh activation function and judiciously choose the internal weights in a data-driven manner such that the resulting features explore the nonlinear, non-saturated regions of the activation function. We introduce skip connections and construct a deep variant of random feature maps by combining several units. To mitigate the curse of dimensionality, we introduce localization where we learn local maps, employing conditional independence. Our modified random feature maps provide excellent forecasting skill for both single trajectory forecasts as well as long-time estimates of statistical properties, for a range of chaotic dynamical systems with dimensions up to 512. In contrast to other methods such as reservoir computers which require extensive hyperparameter tuning, we effectively need to tune only a single hyperparameter, and are able to achieve state-of-the-art forecast skill with much smaller networks.