# Learning Fast and Slow for Online Time Series Forecasting

arXiv: https://arxiv.org/abs/2202.11672 · HTML: https://mlsanity.com/paper/2202.11672/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 76 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [Latent ODEs for Irregularly-Sampled Time Series](https://mlsanity.com/paper/1907.03907/) (2019)
- [CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting](https://mlsanity.com/paper/2202.01575/) (2022)
- [Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System](https://mlsanity.com/paper/2201.12604/) (2022)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data](https://mlsanity.com/paper/2108.09020/) (2021)
- [Learning to Remember Rare Events](https://mlsanity.com/paper/1703.03129/) (2017)
- [Learning from Irregularly-Sampled Time Series: A Missing Data Perspective](https://mlsanity.com/paper/2008.07599/) (2020)
- [DualNet: Continual Learning, Fast and Slow](https://mlsanity.com/paper/2110.00175/) (2021)

## Built on by (most cited first, 30 of 45)

- [Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors](https://mlsanity.com/paper/2305.18803/) (2023)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling](https://mlsanity.com/paper/2309.12659/) (2023)
- [Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretability](https://mlsanity.com/paper/2406.02496/) (2024)
- [On the Resurgence of Recurrent Models for Long Sequences -- Survey and Research Opportunities in the Transformer Era](https://mlsanity.com/paper/2402.08132/) (2024)
- [Lightweight Online Adaption for Time Series Foundation Model Forecasts](https://mlsanity.com/paper/2502.12920/) (2025)
- [AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges](https://mlsanity.com/paper/2304.04661/) (2023)
- [Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adapt](https://mlsanity.com/paper/2403.14949/) (2024)
- [The Forecast After the Forecast: A Post-Processing Shift in Time Series](https://mlsanity.com/paper/2601.20280/) (2026)
- [Spatiotemporal Covariance Neural Networks](https://mlsanity.com/paper/2409.10068/) (2024)
- [Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting](https://mlsanity.com/paper/2506.00635/) (2025)
- [Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models](https://mlsanity.com/paper/2606.14222/) (2026)
- [FrAug: Frequency Domain Augmentation for Time Series Forecasting](https://mlsanity.com/paper/2302.09292/) (2023)
- [Selective Learning for Deep Time Series Forecasting](https://mlsanity.com/paper/2510.25207/) (2025)
- [Act Now: A Novel Online Forecasting Framework for Large-Scale Streaming Data](https://mlsanity.com/paper/2412.00108/) (2024)
- [Continually learning out-of-distribution spatiotemporal data for robust energy forecasting](https://mlsanity.com/paper/2306.06385/) (2023)
- [A Novel Hyperdimensional Computing Framework for Online Time Series Forecasting on the Edge](https://mlsanity.com/paper/2402.01999/) (2024)
- [Towards Principled Test-Time Adaptation for Time Series Forecasting](https://mlsanity.com/paper/2605.17250/) (2026)
- [Towards Foundation Time Series Model: To Synthesize Or Not To Synthesize?](https://mlsanity.com/paper/2403.02534/) (2024)
- [Online time series prediction using feature adjustment](https://mlsanity.com/paper/2509.03810/) (2025)
- [Pre-training Epidemic Time Series Forecasters with Compartmental Prototypes](https://mlsanity.com/paper/2502.03393/) (2025)
- [Continuous Evolution Pool: Taming Recurring Concept Drift in Online Time Series Forecasting](https://mlsanity.com/paper/2506.14790/) (2025)
- [Navigating Out-of-Distribution Electricity Load Forecasting during COVID-19: Benchmarking energy load forecasting models without and with continual learning](https://mlsanity.com/paper/2309.04296/) (2023)
- [PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis](https://mlsanity.com/paper/2405.14650/) (2024)
- [Evolving Multi-Scale Normalization for Time Series Forecasting under Distribution Shifts](https://mlsanity.com/paper/2409.19718/) (2024)
- [Online Time Series Forecasting with Theoretical Guarantees](https://mlsanity.com/paper/2510.18281/) (2025)
- [TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series](https://mlsanity.com/paper/2602.22520/) (2026)
- [Hinge-FM2I: An Approach using Image Inpainting for Interpolating Missing Data in Univariate Time Series](https://mlsanity.com/paper/2406.06631/) (2024)
- [Wormhole: Concept-Aware Deep Representation Learning for Co-Evolving Sequences](https://mlsanity.com/paper/2409.13857/) (2024)
- [Distribution-aware Online Continual Learning for Urban Spatio-Temporal Forecasting](https://mlsanity.com/paper/2411.15893/) (2024)

## Abstract (arXiv)

The fast adaptation capability of deep neural networks in non-stationary environments is critical for online time series forecasting. Successful solutions require handling changes to new and recurring patterns. However, training deep neural forecaster on the fly is notoriously challenging because of their limited ability to adapt to non-stationary environments and the catastrophic forgetting of old knowledge. In this work, inspired by the Complementary Learning Systems (CLS) theory, we propose Fast and Slow learning Networks (FSNet), a holistic framework for online time-series forecasting to simultaneously deal with abrupt changing and repeating patterns. Particularly, FSNet improves the slowly-learned backbone by dynamically balancing fast adaptation to recent changes and retrieving similar old knowledge. FSNet achieves this mechanism via an interaction between two complementary components of an adapter to monitor each layer's contribution to the lost, and an associative memory to support remembering, updating, and recalling repeating events. Extensive experiments on real and synthetic datasets validate FSNet's efficacy and robustness to both new and recurring patterns. Our code is available at urlhttps://github.com/salesforce/fsnet.