# QuitoBench: A High-Quality Open Time Series Forecasting Benchmark

arXiv: https://arxiv.org/abs/2603.26017 · HTML: https://mlsanity.com/paper/2603.26017/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [Chronos-2: From Univariate to Universal Forecasting](https://mlsanity.com/paper/2510.15821/) (2025)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [EasyTPP: Towards Open Benchmarking Temporal Point Processes](https://mlsanity.com/paper/2307.08097/) (2023)
- [FAMMA: A Benchmark for Financial Domain Multilingual Multimodal Question Answering](https://mlsanity.com/paper/2410.04526/) (2024)
- [WeaverBird: Empowering Financial Decision-Making with Large Language Model, Knowledge Base, and Search Engine](https://mlsanity.com/paper/2308.05361/) (2023)
- [Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges](https://mlsanity.com/paper/2510.13654/) (2025)
- [The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models](https://mlsanity.com/paper/2510.23191/) (2025)
- [LookBench: A Live and Holistic Open Benchmark for Fashion Image Retrieval](https://mlsanity.com/paper/2601.14706/) (2026)

## Built on by (most cited first, 10 of 10)

- [Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modeling](https://mlsanity.com/paper/2605.27286/) (2026)
- [Into the ORBIT for Time Series: Training Regimes for Foundation Models](https://mlsanity.com/paper/2608.13262/) (2026)
- [Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context](https://mlsanity.com/paper/2609.15087/) (2026)
- [Beyond Retrieval: A Multitask Benchmark and Model for Code Search](https://mlsanity.com/paper/2605.04615/) (2026)
- [Make LLM Learn to Synthesize from Streaming Experiences through Feedback](https://mlsanity.com/paper/2605.29940/) (2026)
- [ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails](https://mlsanity.com/paper/2605.31073/) (2026)
- [ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker](https://mlsanity.com/paper/2609.31002/) (2026)
- [MixBench-TS: A Multivariate Time Series Forecasting Benchmark Where Channel Mixing Pays Off](https://mlsanity.com/paper/2609.32656/) (2026)
- [A Unified Scaling Law for Time Series Foundation Models](https://mlsanity.com/paper/2610.05269/) (2026)
- [RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecasting](https://mlsanity.com/paper/2610.11164/) (2026)

## Abstract (arXiv)

Time series forecasting is critical across finance, healthcare, and cloud computing, yet progress is constrained by a fundamental bottleneck: the scarcity of large-scale, high-quality benchmarks. To address this gap, we introduce textscQuitoBench, a regime-balanced benchmark for time series forecasting with coverage across eight trend×seasonality×forecastability (TSF) regimes, designed to capture forecasting-relevant properties rather than application-defined domain labels. The benchmark is built upon textscQuito, a billion-scale time series corpus of application traffic from Alipay spanning nine business domains. Benchmarking 10 models from deep learning, foundation models, and statistical baselines across 232,200 evaluation instances, we report four key findings: (i) a context-length crossover where deep learning models lead at short context (L=96) but foundation models dominate at long context (L ≥ 576); (ii) forecastability is the dominant difficulty driver, producing a 3.64 × MAE gap across regimes; (iii) deep learning models match or surpass foundation models at 59 × fewer parameters; and (iv) scaling the amount of training data provides substantially greater benefit than scaling model size for both model families. These findings are validated by strong cross-benchmark and cross-metric consistency. Our open-source release enables reproducible, regime-aware evaluation for time series forecasting research.