# Flipped Classroom: Effective Teaching for Time Series Forecasting

arXiv: https://arxiv.org/abs/2210.08959 · HTML: https://mlsanity.com/paper/2210.08959/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 73 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Generalization in Generation: A closer look at Exposure Bias](https://mlsanity.com/paper/1910.00292/) (2019)
- [AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks](https://mlsanity.com/paper/1902.09689/) (2019)
- [Lipschitz Recurrent Neural Networks](https://mlsanity.com/paper/2006.12070/) (2020)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Comparative Study of CNN and RNN for Natural Language Processing](https://mlsanity.com/paper/1702.01923/) (2017)
- [Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies](https://mlsanity.com/paper/2010.00951/) (2020)
- [Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2001.01520/) (2020)
- [Scheduled Sampling for Transformers](https://mlsanity.com/paper/1906.07651/) (2019)
- [Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI](https://mlsanity.com/paper/1902.07186/) (2019)
- [Long Expressive Memory for Sequence Modeling](https://mlsanity.com/paper/2110.04744/) (2021)
- [A New GAN-based End-to-End TTS Training Algorithm](https://mlsanity.com/paper/1904.04775/) (2019)
- [Teacher-Student Training for Robust Tacotron-based TTS](https://mlsanity.com/paper/1911.02839/) (2019)
- [Direct data-driven forecast of local turbulent heat flux in Rayleigh-B'enard convection](https://mlsanity.com/paper/2202.13129/) (2022)
- [Language Modelling for Sound Event Detection with Teacher Forcing and Scheduled Sampling](https://mlsanity.com/paper/1907.08506/) (2019)
- [Attention Forcing for Sequence-to-sequence Model Training](https://mlsanity.com/paper/1909.12289/) (2019)
- [Fast Modeling and Understanding Fluid Dynamics Systems with Encoder-Decoder Networks](https://mlsanity.com/paper/2006.05409/) (2020)

## Built on by (most cited first, 12 of 12)

- [Learning from Predictions: Fusing Training and Autoregressive Inference for Long-Term Spatiotemporal Forecasts](https://mlsanity.com/paper/2302.11101/) (2023)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [Early Prediction of Causes (not Effects) in Healthcare by Long-Term Clinical Time Series Forecasting](https://mlsanity.com/paper/2408.03816/) (2024)
- [RefreshNet: Learning Multiscale Dynamics through Hierarchical Refreshing](https://mlsanity.com/paper/2401.13282/) (2024)
- [DUALFloodGNN: Physics-informed Graph Neural Network for Operational Flood Modeling](https://mlsanity.com/paper/2512.23964/) (2025)
- [Temporal Convolution Derived Multi-Layered Reservoir Computing](https://mlsanity.com/paper/2407.06771/) (2024)
- [Exposure Bias as Epistemic Underidentification in Recursive Forecasting](https://mlsanity.com/paper/2606.12990/) (2026)
- [Slim multi-scale convolutional autoencoder-based reduced-order models for interpretable features of a complex dynamical system](https://mlsanity.com/paper/2501.03070/) (2025)
- [Mitigating Exposure Bias in Risk-Aware Time Series Forecasting with Soft Tokens](https://mlsanity.com/paper/2512.10056/) (2025)
- [Dual-Criterion Curriculum Learning: Application to Temporal Data](https://mlsanity.com/paper/2603.23573/) (2026)
- [Correct then Forecast: Observer State-Space Models for Time Series Forecasting](https://mlsanity.com/paper/2609.33566/) (2026)
- [NV-Tesseract-AD: Curriculum-Enhanced Conditional Diffusion for Multivariate Time-Series Anomaly Detection](https://mlsanity.com/paper/2610.12754/) (2026)

## Abstract (arXiv)

Sequence-to-sequence models based on LSTM and GRU are a most popular choice for forecasting time series data reaching state-of-the-art performance. Training such models can be delicate though. The two most common training strategies within this context are teacher forcing (TF) and free running (FR). TF can be used to help the model to converge faster but may provoke an exposure bias issue due to a discrepancy between training and inference phase. FR helps to avoid this but does not necessarily lead to better results, since it tends to make the training slow and unstable instead. Scheduled sampling was the first approach tackling these issues by picking the best from both worlds and combining it into a curriculum learning (CL) strategy. Although scheduled sampling seems to be a convincing alternative to FR and TF, we found that, even if parametrized carefully, scheduled sampling may lead to premature termination of the training when applied for time series forecasting. To mitigate the problems of the above approaches we formalize CL strategies along the training as well as the training iteration scale. We propose several new curricula, and systematically evaluate their performance in two experimental sets. For our experiments, we utilize six datasets generated from prominent chaotic systems. We found that the newly proposed increasing training scale curricula with a probabilistic iteration scale curriculum consistently outperforms previous training strategies yielding an NRMSE improvement of up to 81% over FR or TF training. For some datasets we additionally observe a reduced number of training iterations. We observed that all models trained with the new curricula yield higher prediction stability allowing for longer prediction horizons.