# Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems

arXiv: https://arxiv.org/abs/2207.00521 · HTML: https://mlsanity.com/paper/2207.00521/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 89 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 8 of 8)

- [Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model](https://mlsanity.com/paper/1803.04779/) (2018)
- [Machine learning prediction of critical transition and system collapse](https://mlsanity.com/paper/2012.01545/) (2020)
- [Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers](https://mlsanity.com/paper/1910.00659/) (2019)
- [Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography](https://mlsanity.com/paper/1802.02844/) (2018)
- [Rapid Time Series Prediction with a Hardware-Based Reservoir Computer](https://mlsanity.com/paper/1807.07627/) (2018)
- [EM-like Learning Chaotic Dynamics from Noisy and Partial Observations](https://mlsanity.com/paper/1903.10335/) (2019)
- [Effective models and predictability of chaotic multiscale systems via machine learning](https://mlsanity.com/paper/2007.08634/) (2020)
- [Can neural networks predict dynamics they have never seen?](https://mlsanity.com/paper/2111.06783/) (2021)

## Built on by (most cited first, 30 of 33)

- [Generalized Teacher Forcing for Learning Chaotic Dynamics](https://mlsanity.com/paper/2306.04406/) (2023)
- [Out-of-Domain Generalization in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2402.18377/) (2024)
- [Predicting Ordinary Differential Equations with Transformers](https://mlsanity.com/paper/2307.12617/) (2023)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [Bifurcations and loss jumps in RNN training](https://mlsanity.com/paper/2310.17561/) (2023)
- [Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics](https://mlsanity.com/paper/2212.07892/) (2022)
- [Predicting discrete-time bifurcations with deep learning](https://mlsanity.com/paper/2303.09669/) (2023)
- [Catch-22s of reservoir computing](https://mlsanity.com/paper/2210.10211/) (2022)
- [Extrapolating tipping points and simulating non-stationary dynamics of complex systems using efficient machine learning](https://mlsanity.com/paper/2312.06283/) (2023)
- [Attractor reconstruction with reservoir computers: The effect of the reservoir's conditional Lyapunov exponents on faithful attractor reconstruction](https://mlsanity.com/paper/2401.00885/) (2023)
- [Tipping Point Forecasting in Non-Stationary Dynamics on Function Spaces](https://mlsanity.com/paper/2308.08794/) (2023)
- [Deep Learning for predicting rate-induced tipping](https://mlsanity.com/paper/2409.07590/) (2024)
- [Neural Ordinary Differential Equations for Learning and Extrapolating System Dynamics Across Bifurcations](https://mlsanity.com/paper/2507.19036/) (2025)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [Detecting Invariant Manifolds in ReLU-Based RNNs](https://mlsanity.com/paper/2510.03814/) (2025)
- [Reservoir Computing with Error Correction: Long-term Behaviors of Stochastic Dynamical Systems](https://mlsanity.com/paper/2305.00669/) (2023)
- [Deep learning for predicting the occurrence of tipping points](https://mlsanity.com/paper/2407.18693/) (2024)
- [Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics](https://mlsanity.com/paper/2507.06050/) (2025)
- [Suppressing unknown disturbances to dynamical systems using machine learning](https://mlsanity.com/paper/2307.03690/) (2023)
- [Tipping Points of Evolving Epidemiological Networks: Machine Learning-Assisted, Data-Driven Effective Modeling](https://mlsanity.com/paper/2311.00797/) (2023)
- [Changes from Classical Statistics to Modern Statistics and Data Science](https://mlsanity.com/paper/2211.03756/) (2022)
- [Detecting disturbances in network-coupled dynamical systems with machine learning](https://mlsanity.com/paper/2307.12771/) (2023)
- [Fold Bifurcation Identification through Scientific Machine Learning](https://mlsanity.com/paper/2312.14210/) (2023)
- [Machine-learning prediction of tipping with applications to the Atlantic Meridional Overturning Circulation](https://mlsanity.com/paper/2402.14877/) (2024)
- [Prediction Beyond the Medium Range with an Atmosphere-Ocean Model that Combines Physics-based Modeling and Machine Learning](https://mlsanity.com/paper/2405.19518/) (2024)
- [On the dimension of pullback attractors in recurrent neural networks](https://mlsanity.com/paper/2501.11357/) (2025)
- [Attractor learning for spatiotemporally chaotic dynamical systems using echo state networks with transfer learning](https://mlsanity.com/paper/2505.24099/) (2025)
- [What Neuroscience Can Teach AI About Learning in Continuously Changing Environments](https://mlsanity.com/paper/2507.02103/) (2025)
- [Anticipating tipping in spatiotemporal systems with machine learning](https://mlsanity.com/paper/2604.06454/) (2026)
- [Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction](https://mlsanity.com/paper/2605.12683/) (2026)

## Abstract (arXiv)

In this paper we consider the machine learning (ML) task of predicting tipping point transitions and long-term post-tipping-point behavior associated with the time evolution of an unknown (or partially unknown), non-stationary, potentially noisy and chaotic, dynamical system. We focus on the particularly challenging situation where the past dynamical state time series that is available for ML training predominantly lies in a restricted region of the state space, while the behavior to be predicted evolves on a larger state space set not fully observed by the ML model during training. In this situation, it is required that the ML prediction system have the ability to extrapolate to different dynamics past that which is observed during training. We investigate the extent to which ML methods are capable of accomplishing useful results for this task, as well as conditions under which they fail. In general, we found that the ML methods were surprisingly effective even in situations that were extremely challenging, but do (as one would expect) fail when ``too much" extrapolation is required. For the latter case, we investigate the effectiveness of combining the ML approach with conventional modeling based on scientific knowledge, thus forming a hybrid prediction system which we find can enable useful prediction even when its ML-based and knowledge-based components fail when acting alone. We also found that achieving useful results may require using very carefully selected ML hyperparameters and we propose a hyperparameter optimization strategy to address this problem. The main conclusion of this paper is that ML-based approaches are promising tools for predicting the behavior of non-stationary dynamical systems even in the case where the future evolution (perhaps due to the crossing of a tipping point) includes dynamics on a set outside of that explored by the training data.