# Improving Long-term Autoregressive Spatiotemporal Predictions: A Proof of Concept with Fluid Dynamics

arXiv: https://arxiv.org/abs/2508.18565 · HTML: https://mlsanity.com/paper/2508.18565/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [DSformer: A Double Sampling Transformer for Multivariate Time Series Long-term Prediction](https://mlsanity.com/paper/2308.03274/) (2023)
- [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](https://mlsanity.com/paper/2103.12057/) (2021)
- [Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks](https://mlsanity.com/paper/2302.14311/) (2023)
- [Learning Controllable Adaptive Simulation for Multi-resolution Physics](https://mlsanity.com/paper/2305.01122/) (2023)
- [Real-time simulation of parameter-dependent fluid flows through deep learning-based reduced order models](https://mlsanity.com/paper/2106.05722/) (2021)
- [Machine learning for modelling unstructured grid data in computational physics: a review](https://mlsanity.com/paper/2502.09346/) (2025)
- [Learning from Predictions: Fusing Training and Autoregressive Inference for Long-Term Spatiotemporal Forecasts](https://mlsanity.com/paper/2302.11101/) (2023)
- [textitFastSVD-ML-ROM: A Reduced-Order Modeling Framework based on Machine Learning for Real-Time Applications](https://mlsanity.com/paper/2207.11842/) (2022)
- [Multi-fidelity physics constrained neural networks for dynamical systems](https://mlsanity.com/paper/2402.02031/) (2024)
- [Dynamical system prediction from sparse observations using deep neural networks with Voronoi tessellation and physics constraint](https://mlsanity.com/paper/2409.00458/) (2024)
- [Dateformer: Time-modeling Transformer for Longer-term Series Forecasting](https://mlsanity.com/paper/2207.05397/) (2022)
- [Multi-Step Embed to Control: A Novel Deep Learning-based Approach for Surrogate Modelling in Reservoir Simulation](https://mlsanity.com/paper/2409.09920/) (2024)

## Built on by (most cited first, 4 of 4)

- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts](https://mlsanity.com/paper/2605.30184/) (2026)
- [Attention mechanism for scalable mesh-based neural surrogates of free-surface fluids](https://mlsanity.com/paper/2606.23251/) (2026)
- [Cost-Accuracy Trade-offs: Neural Operator vs Classical Numerical Solver](https://mlsanity.com/paper/2609.24021/) (2026)

## Abstract (arXiv)

Data-driven methods are emerging as efficient alternatives to traditional numerical forecasting, offering fast inference and lower computational cost. Yet, for complex systems, long-term accuracy often deteriorates due to error accumulation, and autoregressive training (though effective) demands large GPU memory and may sacrifice short-term performance. We propose the Stochastic PushForward (SPF) framework, which retains one-step-ahead training while enabling multi-step learning. SPF builds a supplementary dataset from model predictions and combines it with ground truth via a stochastic acquisition strategy, balancing short- and long-term performance while reducing overfitting. Multi-step predictions are precomputed between epochs, keeping memory usage stable without storing full unrolled sequences. Experiments on the Burgers' equation and the Shallow Water benchmark show that SPF achieves higher long-term accuracy than autoregressive methods while lowering memory requirements, making it promising for resource-limited and complex simulations.