Test-Time Efficient Pretrained Model Portfolios for Time Series Forecasting
Not on the map. Why?
Abstract (arXiv)
Is bigger always better for time series foundation models? With the question in mind, we explore an alternative to training a single, large monolithic model: building a portfolio of smaller, pretrained forecasting models. By applying ensembling or model selection over these portfolios, we achieve competitive performance on large-scale benchmarks using much fewer parameters. We explore strategies for designing such portfolios and find that collections of specialist models consistently outperform portfolios of independently trained generalists. Remarkably, we demonstrate that post-training a base model is a compute-effective approach for creating sufficiently diverse specialists, and provide evidences that ensembling and model selection are more compute-efficient than test-time fine-tuning.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 7 Oct 2025.
Builds on
33 of 52 references.
References · 33
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
- 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
- 2024Chronos: Learning the Language of Time Seriescited by 759
- 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
- 2023A decoder-only foundation model for time-series forecastingcited by 692
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
21 more
- 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
- 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
- 2021Monash Time Series Forecasting Archivecited by 267
- 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
- 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 139
- 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
- 2020Deep Learning for Time Series Forecasting: Tutorial and Literature Surveycited by 117
- 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
- 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
- 2023A Unified Theory of Diversity in Ensemble Learningcited by 71
- 2024Scaling Laws in Linear Regression: Compute, Parameters, and Datacited by 57
- 2020Neural Ensemble Search for Uncertainty Estimation and Dataset Shiftcited by 56
- 2024Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimizationcited by 56
- 2022Criteria for Classifying Forecasting Methodscited by 48
- 2023AutoGluon-TimeSeries: AutoML for Probabilistic Time Series Forecastingcited by 36
- 2022Forecast combinations: an over 50-year reviewcited by 34
- 2024Scaling Law for Time Series Forecastingcited by 33
- 2024Scaling-laws for Large Time-series Modelscited by 18
- 2022Efficient Automated Deep Learning for Time Series Forecastingcited by 14
- 2024DAM: Towards A Foundation Model for Time Series Forecastingcited by 10
- 2024Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysiscited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.