# ForecastPFN: Synthetically-Trained Zero-Shot Forecasting

arXiv: https://arxiv.org/abs/2311.01933 · HTML: https://mlsanity.com/paper/2311.01933/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 63 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)
- [Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2310.08278/) (2023)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [Meta-learning framework with applications to zero-shot time-series forecasting](https://mlsanity.com/paper/2002.02887/) (2020)
- [Statistical Foundations of Prior-Data Fitted Networks](https://mlsanity.com/paper/2305.11097/) (2023)
- [Transfer learning for time series classification](https://mlsanity.com/paper/1811.01533/) (2018)

## Built on by (most cited first, 30 of 104)

- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [Do-PFN: In-Context Learning for Causal Effect Estimation](https://mlsanity.com/paper/2506.06039/) (2025)
- [TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks](https://mlsanity.com/paper/2402.11137/) (2024)
- [Retrieval Augmented Time Series Forecasting](https://mlsanity.com/paper/2411.08249/) (2024)
- [Is Mamba Capable of In-Context Learning?](https://mlsanity.com/paper/2402.03170/) (2024)
- [Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders](https://mlsanity.com/paper/2405.15273/) (2024)
- [In-Context Fine-Tuning for Time-Series Foundation Models](https://mlsanity.com/paper/2410.24087/) (2024)
- [LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law](https://mlsanity.com/paper/2402.00795/) (2024)
- [Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer](https://mlsanity.com/paper/2405.17478/) (2024)
- [Large language models can be zero-shot anomaly detectors for time series?](https://mlsanity.com/paper/2405.14755/) (2024)
- [TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data](https://mlsanity.com/paper/2502.16294/) (2025)
- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling](https://mlsanity.com/paper/2603.04791/) (2026)
- [In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization](https://mlsanity.com/paper/2404.16795/) (2024)
- [Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data](https://mlsanity.com/paper/2411.10634/) (2024)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2405.02358/) (2024)
- [In-context Time Series Predictor](https://mlsanity.com/paper/2405.14982/) (2024)
- [How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook](https://mlsanity.com/paper/2503.11835/) (2025)
- [Position: The Future of Bayesian Prediction Is Prior-Fitted](https://mlsanity.com/paper/2505.23947/) (2025)
- [In-Context Data Distillation with TabPFN](https://mlsanity.com/paper/2402.06971/) (2024)
- [Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning](https://mlsanity.com/paper/2505.11349/) (2025)
- [Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models](https://mlsanity.com/paper/2410.09385/) (2024)
- [TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting](https://mlsanity.com/paper/2510.25502/) (2025)
- [TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models](https://mlsanity.com/paper/2409.16118/) (2024)

## Abstract (arXiv)

The vast majority of time-series forecasting approaches require a substantial training dataset. However, many real-life forecasting applications have very little initial observations, sometimes just 40 or fewer. Thus, the applicability of most forecasting methods is restricted in data-sparse commercial applications. While there is recent work in the setting of very limited initial data (so-called `zero-shot' forecasting), its performance is inconsistent depending on the data used for pretraining. In this work, we take a different approach and devise ForecastPFN, the first zero-shot forecasting model trained purely on a novel synthetic data distribution. ForecastPFN is a prior-data fitted network, trained to approximate Bayesian inference, which can make predictions on a new time series dataset in a single forward pass. Through extensive experiments, we show that zero-shot predictions made by ForecastPFN are more accurate and faster compared to state-of-the-art forecasting methods, even when the other methods are allowed to train on hundreds of additional in-distribution data points.