# Domain Adaptation for Time Series Forecasting via Attention Sharing

arXiv: https://arxiv.org/abs/2102.06828 · HTML: https://mlsanity.com/paper/2102.06828/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 77 references that are in the map.
- First posted on arXiv 13 Feb 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)
- [Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1607.03516/) (2016)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [Conditional Time Series Forecasting with Convolutional Neural Networks](https://mlsanity.com/paper/1703.04691/) (2017)
- [Few-Shot Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.02536/) (2017)
- [Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes](https://mlsanity.com/paper/1707.09465/) (2017)
- [Temporal Attentive Alignment for Large-Scale Video Domain Adaptation](https://mlsanity.com/paper/1907.12743/) (2019)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data](https://mlsanity.com/paper/2005.10996/) (2020)
- [Continuously Indexed Domain Adaptation](https://mlsanity.com/paper/2007.01807/) (2020)

## Built on by (most cited first, 30 of 68)

- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Chronos-2: From Univariate to Universal Forecasting](https://mlsanity.com/paper/2510.15821/) (2025)
- [Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors](https://mlsanity.com/paper/2305.18803/) (2023)
- [Domain Adaptation for Time Series Under Feature and Label Shifts](https://mlsanity.com/paper/2302.03133/) (2023)
- [Retrieval Augmented Time Series Forecasting](https://mlsanity.com/paper/2411.08249/) (2024)
- [Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts](https://mlsanity.com/paper/2210.03675/) (2022)
- [Contrastive Learning for Unsupervised Domain Adaptation of Time Series](https://mlsanity.com/paper/2206.06243/) (2022)
- [A Survey of Deep Learning and Foundation Models for Time Series Forecasting](https://mlsanity.com/paper/2401.13912/) (2024)
- [Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting](https://mlsanity.com/paper/2111.06581/) (2021)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data](https://mlsanity.com/paper/2502.16294/) (2025)
- [COLA: Cross-city Mobility Transformer for Human Trajectory Simulation](https://mlsanity.com/paper/2403.01801/) (2024)
- [SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction](https://mlsanity.com/paper/2405.15677/) (2024)
- [Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift](https://mlsanity.com/paper/2204.03342/) (2022)
- [Self-Interpretable Time Series Prediction with Counterfactual Explanations](https://mlsanity.com/paper/2306.06024/) (2023)
- [Multivariate Quantile Function Forecaster](https://mlsanity.com/paper/2202.11316/) (2022)
- [Transferable Time-Series Forecasting under Causal Conditional Shift](https://mlsanity.com/paper/2111.03422/) (2021)
- [Contrastive Domain Adaptation for Time-Series via Temporal Mixup](https://mlsanity.com/paper/2212.01555/) (2022)
- [Label-efficient Time Series Representation Learning: A Review](https://mlsanity.com/paper/2302.06433/) (2023)
- [Robust Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.11910/) (2022)
- [Source-Free Domain Adaptation with Temporal Imputation for Time Series Data](https://mlsanity.com/paper/2307.07542/) (2023)
- [Taxonomy-Structured Domain Adaptation](https://mlsanity.com/paper/2306.07874/) (2023)
- [POND: Multi-Source Time Series Domain Adaptation with Information-Aware Prompt Tuning](https://mlsanity.com/paper/2312.12276/) (2023)
- [Evidentially Calibrated Source-Free Time-Series Domain Adaptation with Temporal Imputation](https://mlsanity.com/paper/2406.02635/) (2024)
- [A Comprehensive Review of Trends, Applications and Challenges In Out-of-Distribution Detection](https://mlsanity.com/paper/2209.12935/) (2022)
- [Cross-Frequency Time Series Meta-Forecasting](https://mlsanity.com/paper/2302.02077/) (2023)
- [Cross-Domain Transfer Learning using Attention Latent Features for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2411.06087/) (2024)
- [NuwaTS: a Foundation Model Mending Every Incomplete Time Series](https://mlsanity.com/paper/2405.15317/) (2024)
- [Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement](https://mlsanity.com/paper/2410.02147/) (2024)
- [Beyond Data Scarcity: A Frequency-Driven Framework for Zero-Shot Forecasting](https://mlsanity.com/paper/2411.15743/) (2024)

## Abstract (arXiv)

Recently, deep neural networks have gained increasing popularity in the field of time series forecasting. A primary reason for their success is their ability to effectively capture complex temporal dynamics across multiple related time series. The advantages of these deep forecasters only start to emerge in the presence of a sufficient amount of data. This poses a challenge for typical forecasting problems in practice, where there is a limited number of time series or observations per time series, or both. To cope with this data scarcity issue, we propose a novel domain adaptation framework, Domain Adaptation Forecaster (DAF). DAF leverages statistical strengths from a relevant domain with abundant data samples (source) to improve the performance on the domain of interest with limited data (target). In particular, we use an attention-based shared module with a domain discriminator across domains and private modules for individual domains. We induce domain-invariant latent features (queries and keys) and retrain domain-specific features (values) simultaneously to enable joint training of forecasters on source and target domains. A main insight is that our design of aligning keys allows the target domain to leverage source time series even with different characteristics. Extensive experiments on various domains demonstrate that our proposed method outperforms state-of-the-art baselines on synthetic and real-world datasets, and ablation studies verify the effectiveness of our design choices.