# Model scale versus domain knowledge in statistical forecasting of chaotic systems

arXiv: https://arxiv.org/abs/2303.08011 · HTML: https://mlsanity.com/paper/2303.08011/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 107 references that are in the map.
- First posted on arXiv 13 Mar 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [Temporal Convolutional Networks: A Unified Approach to Action Segmentation](https://mlsanity.com/paper/1608.08242/) (2016)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting](https://mlsanity.com/paper/2201.12886/) (2022)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers](https://mlsanity.com/paper/2007.01547/) (2020)
- [An Analysis of Deep Neural Network Models for Practical Applications](https://mlsanity.com/paper/1605.07678/) (2016)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [sktime: A Unified Interface for Machine Learning with Time Series](https://mlsanity.com/paper/1909.07872/) (2019)
- [Darts: User-Friendly Modern Machine Learning for Time Series](https://mlsanity.com/paper/2110.03224/) (2021)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Do We Really Need Deep Learning Models for Time Series Forecasting?](https://mlsanity.com/paper/2101.02118/) (2021)
- [Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts](https://mlsanity.com/paper/2210.03675/) (2022)
- [Model-free prediction of spatiotemporal dynamical systems with recurrent neural networks: Role of network spectral radius](https://mlsanity.com/paper/1910.04426/) (2019)
- [Highly comparative time-series analysis: The empirical structure of time series and their methods](https://mlsanity.com/paper/1304.1209/) (2013)
- [Benchmarking sparse system identification with low-dimensional chaos](https://mlsanity.com/paper/2302.10787/) (2023)
- [Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning](https://mlsanity.com/paper/2002.11167/) (2020)

## Built on by (most cited first, 27 of 27)

- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems](https://mlsanity.com/paper/2304.04640/) (2023)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning](https://mlsanity.com/paper/2505.11349/) (2025)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [Panda: A pretrained forecast model for chaotic dynamics](https://mlsanity.com/paper/2505.13755/) (2025)
- [CausalDynamics: A large-scale benchmark for structural discovery of dynamical causal models](https://mlsanity.com/paper/2505.16620/) (2025)
- [How more data can hurt: Instability and regularization in next-generation reservoir computing](https://mlsanity.com/paper/2407.08641/) (2024)
- [Machine Learning for Predicting Chaotic Systems](https://mlsanity.com/paper/2407.20158/) (2024)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck](https://mlsanity.com/paper/2311.04896/) (2023)
- [Is Flow Matching Just Trajectory Replay for Sequential Data?](https://mlsanity.com/paper/2602.08318/) (2026)
- [Quantum Reservoir Computing and Risk Bounds](https://mlsanity.com/paper/2501.08640/) (2025)
- [Learning Beyond Experience: Generalizing to Unseen State Space with Reservoir Computing](https://mlsanity.com/paper/2506.05292/) (2025)
- [Characterizing control between interacting subsystems with deep Jacobian estimation](https://mlsanity.com/paper/2507.01946/) (2025)
- [FlowMixer: A Depth-Agnostic Neural Architecture for Interpretable Spatiotemporal Forecasting](https://mlsanity.com/paper/2505.16786/) (2025)
- [Learning Chaotic Dynamics with Neuromorphic Network Dynamics](https://mlsanity.com/paper/2506.10773/) (2025)
- [ChaosNexus: A Foundation Model for ODE-based Chaotic System Forecasting with Hierarchical Multi-scale Awareness](https://mlsanity.com/paper/2509.21802/) (2025)
- [Supporting renewable energy planning and operation with data-driven high-resolution ensemble weather forecast](https://mlsanity.com/paper/2505.04396/) (2025)
- [A tensor network approach for chaotic time series prediction](https://mlsanity.com/paper/2505.17740/) (2025)
- [Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamics](https://mlsanity.com/paper/2507.21299/) (2025)
- [Turning Time Series into Algebraic Equations: Symbolic Machine Learning for Interpretable Modeling of Chaotic Time Series](https://mlsanity.com/paper/2603.07261/) (2026)
- [ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics](https://mlsanity.com/paper/2605.09676/) (2026)
- [Deep Learning Surrogates for Emulating Stochastic Climate Tipping Dynamics](https://mlsanity.com/paper/2605.20580/) (2026)
- [Flow map learning in nonlinear vector autoregressive models: influence of the feature-library structure on the training error](https://mlsanity.com/paper/2605.31438/) (2026)
- [A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems](https://mlsanity.com/paper/2607.14937/) (2026)
- [Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems](https://mlsanity.com/paper/2608.29057/) (2026)

## Abstract (arXiv)

Chaos and unpredictability are traditionally synonymous, yet large-scale machine learning methods recently have demonstrated a surprising ability to forecast chaotic systems well beyond typical predictability horizons. However, recent works disagree on whether specialized methods grounded in dynamical systems theory, such as reservoir computers or neural ordinary differential equations, outperform general-purpose large-scale learning methods such as transformers or recurrent neural networks. These prior studies perform comparisons on few individually-chosen chaotic systems, thereby precluding robust quantification of how statistical modeling choices and dynamical invariants of different chaotic systems jointly determine empirical predictability. Here, we perform the largest to-date comparative study of forecasting methods on the classical problem of forecasting chaos: we benchmark 24 state-of-the-art forecasting methods on a crowdsourced database of 135 low-dimensional systems with 17 forecast metrics. We find that large-scale, domain-agnostic forecasting methods consistently produce predictions that remain accurate up to two dozen Lyapunov times, thereby accessing a new long-horizon forecasting regime well beyond classical methods. We find that, in this regime, accuracy decorrelates with classical invariant measures of predictability like the Lyapunov exponent. However, in data-limited settings outside the long-horizon regime, we find that physics-based hybrid methods retain a comparative advantage due to their strong inductive biases.