mlsanityData to 7 Oct 2026

UniCA: Unified Covariate Adaptation for Time Series Foundation Model

arXiv 2506.22039 ↗First posted 27 Jun 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Time Series Foundation Models (TSFMs) have achieved remarkable success through large-scale pretraining. However, their design primarily targets real-valued series, limiting their ability to handle general forecasting tasks involving diverse and often heterogeneous covariates -- such as categorical variables and multimodal data (e.g., images, text) -- which are typically task-specific and difficult to leverage during pretraining. To address this gap, we propose Unified Covariate Adaptation (UniCA), a framework to bridge TSFMs with general covariate-aware forecasting. UniCA first performs covariate homogenization to transform heterogeneous covariates into high-level homogeneous series representations and then fuses them via a unified attention-based fusion mechanism. UniCA is compatible and universal for adaptation with both homogeneous and heterogeneous covariates, incorporating extra covariate information while preserving the generalization ability of TSFMs.Extensive experiments on multiple unimodal and multimodal covariate-aware forecasting benchmarks demonstrate the superiority of UniCA, highlighting the promise of covariate-aware TSFM adaptation in real-world forecasting scenarios.Code: https://github.com/hanlu-nju/UniCA.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 72 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

43 of 72 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Segment Anythingcited by 8,158
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  9. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  10. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  11. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  12. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
31 more
  1. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  2. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  3. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  4. 2024Chronos: Learning the Language of Time Seriescited by 712
  5. 2016Language Modeling with Gated Convolutional Networkscited by 688
  6. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  7. 2023A decoder-only foundation model for time-series forecastingcited by 650
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  9. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  10. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  11. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  12. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
  13. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  14. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 241
  15. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  16. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
  17. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
  18. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
  19. 2020Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmarkcited by 105
  20. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 100
  21. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 93
  22. 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 92
  23. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
  24. 2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 71
  25. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 65
  26. 2021Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSxcited by 63
  27. 2024GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuningcited by 40
  28. 2021Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecastingcited by 32
  29. 2025ChronosX: Adapting Pretrained Time Series Models with Exogenous Variablescited by 21
  30. 2019Textual Data for Time Series Forecastingcited by 3
  31. 2022Multimodal Neural Network For Demand Forecastingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025CoRA: Covariate-Aware Adaptation of Time Series Foundation Modelscited by 5
  2. 2026Post-Training in Time Series Foundation Models: A Unifying Frameworkcited by 3
  3. 2026LightGTS-Cov: Covariate-Enhanced Time Series Forecastingcited by 0
  4. 2026WorldTS: World Modeling for Multimodal Covariate-aware Time Series Forecastingcited by 0