mlsanityData to 7 Oct 2026

Learning from Predictions: Fusing Training and Autoregressive Inference for Long-Term Spatiotemporal Forecasts

arXiv 2302.11101 ↗First posted 22 Feb 2023cs.LG nlin.CD physics.comp-phcited by 11

Not on the map. Why?

Abstract (arXiv)

Recurrent Neural Networks (RNNs) have become an integral part of modeling and forecasting frameworks in areas like natural language processing and high-dimensional dynamical systems such as turbulent fluid flows. To improve the accuracy of predictions, RNNs are trained using the Backpropagation Through Time (BPTT) method to minimize prediction loss. During testing, RNNs are often used in autoregressive scenarios where the output of the network is fed back into the input. However, this can lead to the exposure bias effect, as the network was trained to receive ground-truth data instead of its own predictions. This mismatch between training and testing is compounded when the state distributions are different, and the train and test losses are measured. To address this, previous studies have proposed solutions for language processing networks with probabilistic predictions. Building on these advances, we propose the Scheduled Autoregressive BPTT (BPTT-SA) algorithm for predicting complex systems. Our results show that BPTT-SA effectively reduces iterative error propagation in Convolutional RNNs and Convolutional Autoencoder RNNs, and demonstrate its capabilities in long-term prediction of high-dimensional fluid flows.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 29 of its 67 references that are in the map. First posted on arXiv 22 Feb 2023.

Builds on

29 of 67 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  5. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  6. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  7. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  8. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  9. 2019Machine Learning for Fluid Mechanicscited by 380
  10. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  11. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  12. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
17 more
  1. 2017Deep Learning for Precipitation Nowcasting: A Benchmark and A New Modelcited by 180
  2. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  3. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  4. 2019Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?cited by 140
  5. 2019Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamicscited by 132
  6. 2019Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networkscited by 127
  7. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  8. 2019Improved Conditional VRNNs for Video Predictioncited by 104
  9. 2018Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flowcited by 97
  10. 2020Multiscale Simulations of Complex Systems by Learning their Effective Dynamicscited by 86
  11. 2013Deep AutoRegressive Networkscited by 60
  12. 2020Convolutional Tensor-Train LSTM for Spatio-temporal Learningcited by 54
  13. 2018Exascale Deep Learning for Climate Analyticscited by 28
  14. 2016Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMscited by 21
  15. 2018Machine Learning for Spatiotemporal Sequence Forecasting: A Surveycited by 19
  16. 2022Flipped Classroom: Effective Teaching for Time Series Forecastingcited by 11
  17. 2018Are extreme dissipation events predictable in turbulent fluid flows?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 2; 2026: 6.

11 papers.

  1. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 55
  2. 2023Model scale versus domain knowledge in statistical forecasting of chaotic systemscited by 27
  3. 2025Structured Kolmogorov-Arnold Neural ODEs for Interpretable Learning and Symbolic Discovery of Nonlinear Dynamicscited by 4
  4. 2025Improving Long-term Autoregressive Spatiotemporal Predictions: A Proof of Concept with Fluid Dynamicscited by 4
  5. 2026Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modelingcited by 3
  6. 2024RefreshNet: Learning Multiscale Dynamics through Hierarchical Refreshingcited by 2
  7. 2026Exposure Bias as Epistemic Underidentification in Recursive Forecastingcited by 1
  8. 2026Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstructioncited by 0
  9. 2026Long-horizon prediction of three-dimensional wall-bounded turbulence with CTA-Swin-UNet and resolvent analysiscited by 0
  10. 2026When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismogramscited by 0
  11. 2026TWIG: A Time-Causal Wavelet Operator for Autoregressive Forecasting on Irregular Graphscited by 0