# TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data

arXiv: https://arxiv.org/abs/2502.16294 · HTML: https://mlsanity.com/paper/2502.16294/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 22 Feb 2025.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [ForecastPFN: Synthetically-Trained Zero-Shot Forecasting](https://mlsanity.com/paper/2311.01933/) (2023)
- [Structure Discovery in Nonparametric Regression through Compositional Kernel Search](https://mlsanity.com/paper/1302.4922/) (2013)
- [Meta-learning framework with applications to zero-shot time-series forecasting](https://mlsanity.com/paper/2002.02887/) (2020)
- [Domain Adaptation for Time Series Forecasting via Attention Sharing](https://mlsanity.com/paper/2102.06828/) (2021)
- [Zero-shot and few-shot time series forecasting with ordinal regression recurrent neural networks](https://mlsanity.com/paper/2003.12162/) (2020)

## Built on by (most cited first, 29 of 29)

- [AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery](https://mlsanity.com/paper/2604.05550/) (2026)
- [TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting](https://mlsanity.com/paper/2510.25502/) (2025)
- [Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models](https://mlsanity.com/paper/2503.11411/) (2025)
- [Relational In-Context Learning via Synthetic Pre-training with Structural Prior](https://mlsanity.com/paper/2603.03805/) (2026)
- [Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy](https://mlsanity.com/paper/2509.21190/) (2025)
- [From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection](https://mlsanity.com/paper/2602.03018/) (2026)
- [Interventional Time Series Priors for Causal Foundation Models](https://mlsanity.com/paper/2603.11090/) (2026)
- [Zero-shot Forecasting by Simulation Alone](https://mlsanity.com/paper/2601.00970/) (2026)
- [Towards Continuous-time Causal Foundation Models](https://mlsanity.com/paper/2605.28880/) (2026)
- [Simulation as Supervision: Mechanistic Pretraining for Scientific Discovery](https://mlsanity.com/paper/2507.08977/) (2025)
- [Mantis: A Foundation Model for Mechanistic Disease Forecasting](https://mlsanity.com/paper/2508.12260/) (2025)
- [WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generation](https://mlsanity.com/paper/2510.11839/) (2025)
- [TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification](https://mlsanity.com/paper/2511.19694/) (2025)
- [Post-Training in Time Series Foundation Models: A Unifying Framework](https://mlsanity.com/paper/2607.20002/) (2026)
- [KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learning](https://mlsanity.com/paper/2605.14907/) (2026)
- [The Forecast Critic: Leveraging Large Language Models for Poor Forecast Identification](https://mlsanity.com/paper/2512.12059/) (2025)
- [Zero-shot Multivariate Time Series Forecasting Using Tabular Prior Fitted Networks](https://mlsanity.com/paper/2604.08400/) (2026)
- [In-Context Learning Under Regime Change](https://mlsanity.com/paper/2604.16988/) (2026)
- [Causal Longitudinal Prior-Fitted Networks for Counterfactual Outcome Prediction](https://mlsanity.com/paper/2606.05797/) (2026)
- [Causal Foundation Models](https://mlsanity.com/paper/2609.03003/) (2026)
- [OmniFuser: Adaptive Multimodal Fusion for Service-Oriented Predictive Maintenance](https://mlsanity.com/paper/2511.01320/) (2025)
- [Covariance-Aware Transformers for Quadratic Programming and Decision Making](https://mlsanity.com/paper/2602.14506/) (2026)
- [Dynamic Linear Coregionalization for Realistic Synthetic Multivariate Time Series](https://mlsanity.com/paper/2604.05064/) (2026)
- [When Does Self-Supervised Learning Transfer to Time-Series Tasks?](https://mlsanity.com/paper/2605.19462/) (2026)
- [Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification](https://mlsanity.com/paper/2605.21742/) (2026)
- [REGEN: Reference-Guided Synthetic Multivariate Time Series Generation for Forecasting](https://mlsanity.com/paper/2606.05264/) (2026)
- [Beyond Average Error through Oracle-Informed Stress Tests for Time-Series Forecasting](https://mlsanity.com/paper/2609.22820/) (2026)
- [World2TS: Real-World Event Context for Time-Series Forecasting](https://mlsanity.com/paper/2610.14861/) (2026)
- [Lotus: Learning to Understand Time-series by Learning to Forecast](https://mlsanity.com/paper/2610.18877/) (2026)

## Abstract (arXiv)

The diversity of time series applications and scarcity of domain-specific data highlight the need for time-series models with strong few-shot learning capabilities. In this work, we propose a novel training scheme and a transformer-based architecture, collectively referred to as TimePFN, for multivariate time-series (MTS) forecasting. TimePFN is based on the concept of Prior-data Fitted Networks (PFN), which aims to approximate Bayesian inference. Our approach consists of (1) generating synthetic MTS data through diverse Gaussian process kernels and the linear coregionalization method, and (2) a novel MTS architecture capable of utilizing both temporal and cross-channel dependencies across all input patches. We evaluate TimePFN on several benchmark datasets and demonstrate that it outperforms the existing state-of-the-art models for MTS forecasting in both zero-shot and few-shot settings. Notably, fine-tuning TimePFN with as few as 500 data points nearly matches full dataset training error, and even 50 data points yield competitive results. We also find that TimePFN exhibits strong univariate forecasting performance, attesting to its generalization ability. Overall, this work unlocks the power of synthetic data priors for MTS forecasting and facilitates strong zero- and few-shot forecasting performance.