# AdaRNN: Adaptive Learning and Forecasting of Time Series

arXiv: https://arxiv.org/abs/2108.04443 · HTML: https://mlsanity.com/paper/2108.04443/

## Key facts

- Cited by 121 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 10 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Deep Subdomain Adaptation Network for Image Classification](https://mlsanity.com/paper/2106.09388/) (2021)
- [Visual Domain Adaptation with Manifold Embedded Distribution Alignment](https://mlsanity.com/paper/1807.07258/) (2018)
- [Imaging Time-Series to Improve Classification and Imputation](https://mlsanity.com/paper/1506.00327/) (2015)
- [Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models](https://mlsanity.com/paper/1909.09020/) (2019)
- [Meta-learning framework with applications to zero-shot time-series forecasting](https://mlsanity.com/paper/2002.02887/) (2020)
- [Transfer learning for time series classification](https://mlsanity.com/paper/1811.01533/) (2018)
- [Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data](https://mlsanity.com/paper/1706.03161/) (2017)
- [Transfer Learning with Dynamic Distribution Adaptation](https://mlsanity.com/paper/1909.08531/) (2019)
- [Transfer Learning with Dynamic Adversarial Adaptation Network](https://mlsanity.com/paper/1909.08184/) (2019)
- [Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks](https://mlsanity.com/paper/1703.06345/) (2017)
- [Probabilistic Time Series Forecasting with Structured Shape and Temporal Diversity](https://mlsanity.com/paper/2010.07349/) (2020)
- [Transfer Learning for Sequence Labeling Using Source Model and Target Data](https://mlsanity.com/paper/1902.05309/) (2019)
- [Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks](https://mlsanity.com/paper/1807.01705/) (2018)
- [Transfer Learning for Sequences via Learning to Collocate](https://mlsanity.com/paper/1902.09092/) (2019)

## Built on by (most cited first, 30 of 121)

- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](https://mlsanity.com/paper/2303.14483/) (2023)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [USB: A Unified Semi-supervised Learning Benchmark for Classification](https://mlsanity.com/paper/2208.07204/) (2022)
- [Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting](https://mlsanity.com/paper/2302.14829/) (2023)
- [Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practices](https://mlsanity.com/paper/2203.10716/) (2022)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment](https://mlsanity.com/paper/2309.13378/) (2023)
- [Frequency Adaptive Normalization For Non-stationary Time Series Forecasting](https://mlsanity.com/paper/2409.20371/) (2024)
- [Retrieval Augmented Time Series Forecasting](https://mlsanity.com/paper/2411.08249/) (2024)
- [Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer](https://mlsanity.com/paper/2205.13947/) (2022)
- [ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM](https://mlsanity.com/paper/2506.14766/) (2025)
- [Domain-invariant Feature Exploration for Domain Generalization](https://mlsanity.com/paper/2207.12020/) (2022)
- [Out-of-Distribution Representation Learning for Time Series Classification](https://mlsanity.com/paper/2209.07027/) (2022)
- [WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series](https://mlsanity.com/paper/2203.09978/) (2022)
- [GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization](https://mlsanity.com/paper/2211.02733/) (2022)
- [Domain Generalization through Meta-Learning: A Survey](https://mlsanity.com/paper/2404.02785/) (2024)
- [Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning](https://mlsanity.com/paper/2406.09130/) (2024)
- [Non-stationary Diffusion For Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2505.04278/) (2025)
- [Calibration of Time-Series Forecasting: Detecting and Adapting Context-Driven Distribution Shift](https://mlsanity.com/paper/2310.14838/) (2023)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [Cross-city Few-Shot Traffic Forecasting via Traffic Pattern Bank](https://mlsanity.com/paper/2308.09727/) (2023)
- [DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecasting](https://mlsanity.com/paper/2306.09862/) (2023)
- [Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptation](https://mlsanity.com/paper/2302.02561/) (2023)
- [Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detection](https://mlsanity.com/paper/2306.14728/) (2023)
- [Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts](https://mlsanity.com/paper/2403.05026/) (2024)
- [Proactive Model Adaptation Against Concept Drift for Online Time Series Forecasting](https://mlsanity.com/paper/2412.08435/) (2024)
- [Spatio-Temporal Few-Shot Learning via Diffusive Neural Network Generation](https://mlsanity.com/paper/2402.11922/) (2024)
- [Bi-Mamba+: Bidirectional Mamba for Time Series Forecasting](https://mlsanity.com/paper/2404.15772/) (2024)
- [Continuous Temporal Domain Generalization](https://mlsanity.com/paper/2405.16075/) (2024)

## Abstract (arXiv)

Time series has wide applications in the real world and is known to be difficult to forecast. Since its statistical properties change over time, its distribution also changes temporally, which will cause severe distribution shift problem to existing methods. However, it remains unexplored to model the time series in the distribution perspective. In this paper, we term this as Temporal Covariate Shift (TCS). This paper proposes Adaptive RNNs (AdaRNN) to tackle the TCS problem by building an adaptive model that generalizes well on the unseen test data. AdaRNN is sequentially composed of two novel algorithms. First, we propose Temporal Distribution Characterization to better characterize the distribution information in the TS. Second, we propose Temporal Distribution Matching to reduce the distribution mismatch in TS to learn the adaptive TS model. AdaRNN is a general framework with flexible distribution distances integrated. Experiments on human activity recognition, air quality prediction, and financial analysis show that AdaRNN outperforms the latest methods by a classification accuracy of 2.6% and significantly reduces the RMSE by 9.0%. We also show that the temporal distribution matching algorithm can be extended in Transformer structure to boost its performance.