# In-context Time Series Predictor

arXiv: https://arxiv.org/abs/2405.14982 · HTML: https://mlsanity.com/paper/2405.14982/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [What learning algorithm is in-context learning? Investigations with linear models](https://mlsanity.com/paper/2211.15661/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Trained Transformers Learn Linear Models In-Context](https://mlsanity.com/paper/2306.09927/) (2023)
- [Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection](https://mlsanity.com/paper/2306.04637/) (2023)
- [TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables](https://mlsanity.com/paper/2402.19072/) (2024)
- [Transformers as Algorithms: Generalization and Stability in In-context Learning](https://mlsanity.com/paper/2301.07067/) (2023)
- [Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers](https://mlsanity.com/paper/2212.10559/) (2022)
- [Large Language Models for Time Series: A Survey](https://mlsanity.com/paper/2402.01801/) (2024)
- [ForecastPFN: Synthetically-Trained Zero-Shot Forecasting](https://mlsanity.com/paper/2311.01933/) (2023)
- [How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations](https://mlsanity.com/paper/2310.10616/) (2023)
- [Accurate estimation of influenza epidemics using Google search data via ARGO](https://mlsanity.com/paper/1505.00864/) (2015)
- [ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning](https://mlsanity.com/paper/2310.09488/) (2023)

## Built on by (most cited first, 23 of 23)

- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [HEARTS: Benchmarking LLM Reasoning on Health Time Series](https://mlsanity.com/paper/2603.06638/) (2026)
- [LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuning](https://mlsanity.com/paper/2411.09947/) (2024)
- [Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision](https://mlsanity.com/paper/2506.00807/) (2025)
- [TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification](https://mlsanity.com/paper/2511.19694/) (2025)
- [Rethinking Zero-Shot Time Series Classification: From Task-specific Classifiers to In-Context Inference](https://mlsanity.com/paper/2602.00620/) (2026)
- [Time Series Reasoning via Process-Verifiable Thinking Data Synthesis and Scheduling for Tailored LLM Reasoning](https://mlsanity.com/paper/2602.07830/) (2026)
- [Why Do Time Series Models Need Long Context Windows?](https://mlsanity.com/paper/2606.01999/) (2026)
- [Enhanced Credit Score Prediction Using Ensemble Deep Learning Model](https://mlsanity.com/paper/2410.00256/) (2024)
- [GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning](https://mlsanity.com/paper/2510.04567/) (2025)
- [Free Energy Mixer](https://mlsanity.com/paper/2602.07160/) (2026)
- [A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks](https://mlsanity.com/paper/2603.22586/) (2026)
- [StretchTime: Adaptive Time Series Forecasting via Symplectic Attention](https://mlsanity.com/paper/2602.08983/) (2026)
- [LISA: Laplacian In-context Spectral Analysis](https://mlsanity.com/paper/2602.04906/) (2026)
- [Deep Time-Series Forecasting in 10 Years: A Survey](https://mlsanity.com/paper/2603.19899/) (2026)
- [CAST: Causal Anchored Simplex Transport for Distribution-Valued Time Series](https://mlsanity.com/paper/2605.16919/) (2026)
- [On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series](https://mlsanity.com/paper/2605.26194/) (2026)
- [Trio: Learning Time-Series Forecasting with Temporal-Spatial-Sample Attention and Structural Causal Priors](https://mlsanity.com/paper/2606.07291/) (2026)
- [In-Context Inpainting for Time Series Forecasting](https://mlsanity.com/paper/2608.23855/) (2026)
- [Beyond Average Error through Oracle-Informed Stress Tests for Time-Series Forecasting](https://mlsanity.com/paper/2609.22820/) (2026)
- [KiT: A Foundation Model for Financial Time-Series Forecasting using DiffusionTransformers](https://mlsanity.com/paper/2609.34507/) (2026)
- [Latent Inference-Time Guidance of Time Series Foundation Models](https://mlsanity.com/paper/2609.38058/) (2026)
- [Second-Order Problem Solving for Recursive Self-Improvement in Formal Verification](https://mlsanity.com/paper/2610.05701/) (2026)

## Abstract (arXiv)

Recent Transformer-based large language models (LLMs) demonstrate in-context learning ability to perform various functions based solely on the provided context, without updating model parameters. To fully utilize the in-context capabilities in time series forecasting (TSF) problems, unlike previous Transformer-based or LLM-based time series forecasting methods, we reformulate "time series forecasting tasks" as input tokens by constructing a series of (lookback, future) pairs within the tokens. This method aligns more closely with the inherent in-context mechanisms, and is more parameter-efficient without the need of using pre-trained LLM parameters. Furthermore, it addresses issues such as overfitting in existing Transformer-based TSF models, consistently achieving better performance across full-data, few-shot, and zero-shot settings compared to previous architectures.