mlsanityData to 14 Oct 2026

In-context Time Series Predictor

arXiv 2405.14982 ↗First posted 23 May 2024cs.LG cs.AI cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Recent Transformer-based large language models (LLMs) demonstrate in-context learning ability to perform various functions based solely on the provided context, without updating model parameters. To fully utilize the in-context capabilities in time series forecasting (TSF) problems, unlike previous Transformer-based or LLM-based time series forecasting methods, we reformulate "time series forecasting tasks" as input tokens by constructing a series of (lookback, future) pairs within the tokens. This method aligns more closely with the inherent in-context mechanisms, and is more parameter-efficient without the need of using pre-trained LLM parameters. Furthermore, it addresses issues such as overfitting in existing Transformer-based TSF models, consistently achieving better performance across full-data, few-shot, and zero-shot settings compared to previous architectures.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

30 of 42 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  3. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  4. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  7. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  8. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  9. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  10. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  11. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  12. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
18 more
  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  2. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  3. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  4. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  5. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  6. 2022Transformers in Time Series: A Surveycited by 497
  7. 2023Larger language models do in-context learning differentlycited by 394
  8. 2021Transformers Can Do Bayesian Inferencecited by 360
  9. 2023Trained Transformers Learn Linear Models In-Contextcited by 337
  10. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
  11. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 248
  12. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  13. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  14. 2024Large Language Models for Time Series: A Surveycited by 114
  15. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  16. 2023How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representationscited by 75
  17. 2015Accurate estimation of influenza epidemics using Google search data via ARGOcited by 17
  18. 2023ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 17.

23 papers.

Sort:

By area:

  1. 2024Zero-shot forecasting of chaotic systemscited by 28
  2. 2026HEARTS: Benchmarking LLM Reasoning on Health Time Seriescited by 11
  3. 2024LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuningcited by 10
  4. 2025Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decisioncited by 8
  5. 2025TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classificationcited by 3
  6. 2026Rethinking Zero-Shot Time Series Classification: From Task-specific Classifiers to In-Context Inferencecited by 3
  7. 2026Time Series Reasoning via Process-Verifiable Thinking Data Synthesis and Scheduling for Tailored LLM Reasoningcited by 3
  8. 2026Why Do Time Series Models Need Long Context Windows?cited by 3
  9. 2024Enhanced Credit Score Prediction Using Ensemble Deep Learning Modelcited by 2
  10. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2
  11. 2026Free Energy Mixercited by 2
  12. 2026A Foundation Model for Instruction-Conditioned In-Context Time Series Taskscited by 2
11 more
  1. 2026StretchTime: Adaptive Time Series Forecasting via Symplectic Attentioncited by 1
  2. 2026LISA: Laplacian In-context Spectral Analysiscited by 0
  3. 2026Deep Time-Series Forecasting in 10 Years: A Surveycited by 0
  4. 2026CAST: Causal Anchored Simplex Transport for Distribution-Valued Time Seriescited by 0
  5. 2026On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Seriescited by 0
  6. 2026Trio: Learning Time-Series Forecasting with Temporal-Spatial-Sample Attention and Structural Causal Priorscited by 0
  7. 2026In-Context Inpainting for Time Series Forecastingcited by 0
  8. 2026Beyond Average Error through Oracle-Informed Stress Tests for Time-Series Forecastingcited by 0
  9. 2026KiT: A Foundation Model for Financial Time-Series Forecasting using DiffusionTransformerscited by 0
  10. 2026Latent Inference-Time Guidance of Time Series Foundation Modelscited by 0
  11. 2026Second-Order Problem Solving for Recursive Self-Improvement in Formal Verificationcited by 0