mlsanityData to 14 Oct 2026

GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation

arXiv 2410.10393 ↗First posted 14 Oct 2024cs.LG stat.MLcited by 212

Not on the map. Why?

Abstract (arXiv)

Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. However, the advancement of these models has been hindered by the lack of comprehensive benchmarks. To address this gap, we introduce the General Time Series Forecasting Model Evaluation, GIFT-Eval, a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points, spanning seven domains, 10 frequencies, multivariate inputs, and prediction lengths ranging from short to long-term forecasts. To facilitate the effective pretraining and evaluation of foundation models, we also provide a non-leaking pretraining dataset containing approximately 230 billion data points. Additionally, we provide a comprehensive analysis of 17 baselines, which includes statistical models, deep learning models, and foundation models. We discuss each model in the context of various benchmark characteristics and offer a qualitative analysis that spans both deep learning and foundation models. We believe the insights from this analysis, along with access to this new standard zero-shot time series forecasting benchmark, will guide future developments in time series foundation models. Code, data, and the leaderboard can be found at https://github.com/SalesforceAIResearch/gift-eval .

Key facts. Cited by 212 papers in the map (data through 14 Oct 2026). Builds on 28 of its 52 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

28 of 52 references.

References · 28

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  5. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  7. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  8. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  9. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  10. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  11. 2024Chronos: Learning the Language of Time Seriescited by 759
  12. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
16 more
  1. 2023A decoder-only foundation model for time-series forecastingcited by 692
  2. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  3. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  4. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  5. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  6. 2021Monash Time Series Forecasting Archivecited by 267
  7. 2023LargeST: A Benchmark Dataset for Large-Scale Traffic Forecastingcited by 118
  8. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 117
  9. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  10. 2023ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modelingcited by 47
  11. 2024Scaling Law for Time Series Forecastingcited by 33
  12. 2021Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecastingcited by 32
  13. 2023BuildingsBench: A Large-Scale Dataset of 900K Buildings and Benchmark for Short-Term Load Forecastingcited by 26
  14. 2023Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domaincited by 23
  15. 2023LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Predictioncited by 20
  16. 2023Bridging AI and Energy Forecasting: An Autonomous Workflow with Customized Toolkitcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 59; 2026: 151.

212 papers.

Sort:

By area:

  1. 2025Chronos-2: From Univariate to Universal Forecastingcited by 194
  2. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  3. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
  4. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 69
  5. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 68
  6. 2025fev-bench: A Realistic Benchmark for Time Series Forecastingcited by 64
  7. 2025From Tables to Time: Extending TabPFN-v2 to Time Series Forecastingcited by 60
  8. 2025OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Datacited by 36
  9. 2026It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarkscited by 33
  10. 2026Toto 2.0: Time Series Forecasting Enters the Scaling Eracited by 30
  11. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 28
  12. 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Modelcited by 24
38 more
  1. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  2. 2026Beyond IID: How General Are Tabular Foundation Models, Really?cited by 20
  3. 2024xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memoriescited by 19
  4. 2025TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecastingcited by 17
  5. 2025Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challengescited by 16
  6. 2025CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Datacited by 13
  7. 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecastingcited by 13
  8. 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streamingcited by 13
  9. 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insightscited by 12
  10. 2025VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbonescited by 12
  11. 2025When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inferencecited by 12
  12. 2026Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Modelscited by 12
  13. 2025Performance of Zero-Shot Time Series Foundation Models on Cloud Datacited by 10
  14. 2025From Samples to Scenarios: A New Paradigm for Probabilistic Forecastingcited by 10
  15. 2026QuitoBench: A High-Quality Open Time Series Forecasting Benchmarkcited by 10
  16. 2026Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modelingcited by 10
  17. 2025TimeCopilotcited by 9
  18. 2025Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Modelscited by 9
  19. 2025Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classificationcited by 9
  20. 2026Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecastingcited by 9
  21. 2026Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Modelscited by 9
  22. 2025There are no Champions in Supervised Long-Term Time Series Forecastingcited by 8
  23. 2025Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attentioncited by 8
  24. 2025MoTime: A Dataset Suite for Multimodal Time Series Forecastingcited by 7
  25. 2025TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectivenesscited by 7
  26. 2025UniCA: Unified Covariate Adaptation for Time Series Foundation Modelcited by 7
  27. 2025SEMPO: Lightweight Foundation Models for Time Series Forecastingcited by 7
  28. 2025Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Modelscited by 7
  29. 2026TimeOmni-VL: Unified Models for Time Series Understanding and Generationcited by 7
  30. 2026Rethinking Multimodal Time-Series Forecasting Evaluationcited by 7
  31. 2025Foundation models for time series forecasting: Application in conformal predictioncited by 6
  32. 2025How Foundational are Foundation Models for Time Series Forecasting?cited by 5
  33. 2025Exploring Accuracy Law for Deep Time Series Forecasters: An Empirical Studycited by 5
  34. 2025TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysiscited by 5
  35. 2026Zero-shot Forecasting by Simulation Alonecited by 5
  36. 2026Overcoming the Modality Gap in Context-Aided Forecastingcited by 5
  37. 2026Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecastingcited by 5
  38. 2025GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series Datacited by 4

50 most cited of 212 All 212 (JSON)