mlsanityData to 13 Oct 2026

Unify and Anchor: A Context-Aware Transformer for Cross-Domain Time Series Forecasting

arXiv 2503.01157 ↗First posted 3 Mar 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

The rise of foundation models has revolutionized natural language processing and computer vision, yet their best practices to time series forecasting remains underexplored. Existing time series foundation models often adopt methodologies from these fields without addressing the unique characteristics of time series data. In this paper, we identify two key challenges in cross-domain time series forecasting: the complexity of temporal patterns and semantic misalignment. To tackle these issues, we propose the ``Unify and Anchor" transfer paradigm, which disentangles frequency components for a unified perspective and incorporates external context as domain anchors for guided adaptation. Based on this framework, we introduce ContexTST, a Transformer-based model that employs a time series coordinator for structured representation and the Transformer blocks with a context-informed mixture-of-experts mechanism for effective cross-domain generalization. Extensive experiments demonstrate that ContexTST advances state-of-the-art forecasting performance while achieving strong zero-shot transferability across diverse domains.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 29 of its 60 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

29 of 60 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2024The Llama 3 Herd of Modelscited by 14,486
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  7. 2020Scaling Laws for Neural Language Modelscited by 6,494
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,999
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,463
  10. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,157
  11. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,016
  12. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,665
17 more
  1. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,442
  2. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,206
  3. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 874
  4. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 751
  5. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 623
  6. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 602
  7. 2022Transformers in Time Series: A Surveycited by 493
  8. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 402
  9. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 337
  10. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 246
  11. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 124
  12. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 115
  13. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 92
  14. 2024Context is Key: A Benchmark for Forecasting with Essential Textual Informationcited by 83
  15. 2021Domain Adaptation for Time Series Forecasting via Attention Sharingcited by 68
  16. 2024Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfercited by 32
  17. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 24

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Superposition Is Not Necessary: A Mechanistic Interpretability Analysis of Transformer Representations for Time Series Forecasting
  2. 2025S2TX: Cross-Attention Multi-Scale State-Space Transformer for Time Series Forecasting
  3. 2025OneCast: Structured Decomposition and Modular Generation for Cross-Domain Time Series Forecasting
  4. 2025STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions
  5. 2025Aurora: Towards Universal Generative Multimodal Time Series Forecasting
  6. 2025UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting
  7. 2025Enhancing LLMs for Time Series Forecasting via Structure-Guided Cross-Modal Alignment
  8. 2026MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models