mlsanityData to 14 Oct 2026

Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models

arXiv 2606.14222 ↗First posted 12 Jun 2026cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

The rapid evolution of Time Series Foundation Models (TSFMs) has advanced zero-shot forecasting across diverse domains. Inspired by the current form of Large Language Models, future TSFMs may be offered as commercialized, closed-source API services. However, many existing online adaptation methods still rely on white-box access for parameter fine-tuning or gradient backpropagation. This paradigm mismatch raises a question: In black-box online adaptation for TSFMs, what should we learn? We answer this with an insight: the predictive errors of the base model are conditioned on both the input and output of the base model (i.e., the context of errors). To validate this insight, we propose ORCA (Online Residual Contextual Adaptation). We conduct extensive experiments across 5 state-of-the-art TSFMs and 8 datasets to demonstrate the effectiveness of our approach. Furthermore, through ablation studies, we quantitatively analyze the impact of different adapter learning hypotheses on the final adaptation performance in black-box online adaptation. Code available at https://github.com/Fifthky/ORCA.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 60 of its 82 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

60 of 82 references.

References · 60

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  6. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  7. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  8. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  9. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  10. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  11. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  12. 2024Chronos: Learning the Language of Time Seriescited by 759
38 more
  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  2. 2023A decoder-only foundation model for time-series forecastingcited by 692
  3. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  4. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  5. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  6. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  7. 2024LoRA Learns Less and Forgets Lesscited by 328
  8. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 248
  9. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  10. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  11. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  12. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  13. 2025Chronos-2: From Univariate to Universal Forecastingcited by 194
  14. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  15. 2023TimeGPT-1cited by 158
  16. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  17. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
  18. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 113
  19. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  20. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  21. 2023OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensemblingcited by 74
  22. 2025Nested Learning: The Illusion of Deep Learning Architecturescited by 72
  23. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 69
  24. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecastingcited by 64
  25. 2025fev-bench: A Realistic Benchmark for Time Series Forecastingcited by 64
  26. 2023Continual Learning: Applications and the Road Forwardcited by 59
  27. 2022Learning Fast and Slow for Online Time Series Forecastingcited by 45
  28. 2023Maintaining Plasticity in Deep Continual Learningcited by 43
  29. 2025Battling the Non-stationarity in Time Series Forecasting via Test-time Adaptationcited by 35
  30. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 33
  31. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 28
  32. 2023Calibration of Time-Series Forecasting: Detecting and Adapting Context-Driven Distribution Shiftcited by 25
  33. 2025Lightweight Online Adaption for Time Series Foundation Model Forecastscited by 16
  34. 2025AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecastingcited by 15
  35. 2025Accurate Parameter-Efficient Test-Time Adaptation for Time Series Forecastingcited by 14
  36. 2024Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adaptcited by 13
  37. 2024Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecastingcited by 11
  38. 2025From Samples to Scenarios: A New Paradigm for Probabilistic Forecastingcited by 10

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 1
  2. 2026When Do Corrective Features Help? An Agent for Corrective Feature Discovery on Black-Box Forecasterscited by 0
  3. 2026Systematic Evaluation of TabPFN-TS and Chronos-2 for Zero-Shot Heat Load Forecasting in District Heating Networkscited by 0
  4. 2026Downside-Controlled Online Forecast Combination under Delayed and Revised Outcomescited by 0
  5. 2026Pythia: Toward Foundation World Models for Multimodal Time Seriescited by 0
  6. 2026A Unified Scaling Law for Time Series Foundation Modelscited by 0
  7. 2026The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptationcited by 0
  8. 2026Training on the Future: A Delay-Aware Audit of Test-Time Adaptation for Time-Series Forecastingcited by 0
  9. 2026ECHO: Correcting Time Series Foundation Model Forecasts with Historical Analogues and Errorscited by 0