# Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

arXiv: https://arxiv.org/abs/2305.00100 · HTML: https://mlsanity.com/paper/2305.00100/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 83 references that are in the map.
- First posted on arXiv 28 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere](https://mlsanity.com/paper/2003.11927/) (2020)
- [Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models](https://mlsanity.com/paper/2102.05107/) (2021)
- [Overview frequency principle/spectral bias in deep learning](https://mlsanity.com/paper/2201.07395/) (2022)
- [Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers](https://mlsanity.com/paper/1910.00659/) (2019)
- [Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution](https://mlsanity.com/paper/2304.07029/) (2023)
- [A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/2201.08910/) (2022)
- [Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation](https://mlsanity.com/paper/2109.12269/) (2021)
- [Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computing](https://mlsanity.com/paper/2203.13294/) (2022)
- [Reservoir Computing as a Tool for Climate Predictability Studies](https://mlsanity.com/paper/2103.06206/) (2021)
- [Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks](https://mlsanity.com/paper/2304.12865/) (2023)
- [Hierarchical Architectures in Reservoir Computing Systems](https://mlsanity.com/paper/2105.06923/) (2021)
- [Predicting Shallow Water Dynamics using Echo-State Networks with Transfer Learning](https://mlsanity.com/paper/2112.09182/) (2021)
- [`Next Generation' Reservoir Computing: an Empirical Data-Driven Expression of Dynamical Equations in Time-Stepping Form](https://mlsanity.com/paper/2201.05193/) (2022)
- [Generative Modeling of High-resolution Global Precipitation Forecasts](https://mlsanity.com/paper/2210.12504/) (2022)
- [Investigation of Proper Orthogonal Decomposition for Echo State Networks](https://mlsanity.com/paper/2211.17179/) (2022)

## Built on by (most cited first, 3 of 3)

- [HRRRCast: a data-driven emulator for regional weather forecasting at convection allowing scales](https://mlsanity.com/paper/2507.05658/) (2025)
- [Predicting unobserved climate time series data at distant areas via spatial correlation using reservoir computing](https://mlsanity.com/paper/2406.03061/) (2024)
- [Visibility nowcasting in South Korea: a machine learning approach to class imbalance and distribution shift](https://mlsanity.com/paper/2605.21507/) (2026)

## Abstract (arXiv)

The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use datasets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (1) a form of Nonlinear Vector Autoregression (NVAR), and (2) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi-Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.