# Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates

arXiv: https://arxiv.org/abs/2603.17439 · HTML: https://mlsanity.com/paper/2603.17439/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 47 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [Sundial: A Family of Highly Capable Time Series Foundation Models](https://mlsanity.com/paper/2502.00816/) (2025)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [ForecastPFN: Synthetically-Trained Zero-Shot Forecasting](https://mlsanity.com/paper/2311.01933/) (2023)
- [Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting](https://mlsanity.com/paper/2402.05956/) (2024)
- [XTab: Cross-table Pretraining for Tabular Transformers](https://mlsanity.com/paper/2305.06090/) (2023)
- [TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023](https://mlsanity.com/paper/2307.14338/) (2023)
- [Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs](https://mlsanity.com/paper/2202.08408/) (2022)
- [Moirai 2.0: When Less Is More for Time Series Forecasting](https://mlsanity.com/paper/2511.11698/) (2025)
- [This Time is Different: An Observability Perspective on Time Series Foundation Models](https://mlsanity.com/paper/2505.14766/) (2025)
- [fev-bench: A Realistic Benchmark for Time Series Forecasting](https://mlsanity.com/paper/2509.26468/) (2025)

## Built on by (most cited first, 4 of 4)

- [Deep Time-Series Forecasting in 10 Years: A Survey](https://mlsanity.com/paper/2603.19899/) (2026)
- [Aurora-X: Built for Extreme Time Series Forecasting](https://mlsanity.com/paper/2609.31038/) (2026)
- [XMatch: Enhancing Covariate-Aware Time Series Forecasting through Tree-Structured Exogenous Matching](https://mlsanity.com/paper/2609.34939/) (2026)
- [JudgeCast: Time Series Forecasting with Experience-Informed Covariate Judgements](https://mlsanity.com/paper/2609.36966/) (2026)

## Abstract (arXiv)

Transformers enable in-context learning (ICL) for rapid, gradient-free adaptation in time series forecasting, yet most ICL-style approaches rely on tabularized, hand-crafted features, while end-to-end sequence models lack inference-time adaptation. We bridge this gap with a unified framework, Baguan-TS, which integrates the raw-sequence representation learning with ICL, instantiated by a 3D Transformer that attends jointly over temporal, variable, and context axes. To make this high-capacity model practical, we tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space retrieval-based local calibration; and (ii) output oversmoothing, mitigated via context-overfitting strategy. On public benchmark with covariates, Baguan-TS consistently outperforms established baselines, achieving the highest win rate and significant reductions in both point and probabilistic forecasting metrics. Further evaluations across diverse real-world energy datasets demonstrate its robustness, yielding substantial improvements.