mlsanityData to 14 Oct 2026

How Much Computation Does Every Forecast Need? Structured Block Skipping in Two Time-Series Foundation Models

arXiv 2610.13208 ↗First posted 9 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Time-series foundation models use the same network depth for every input, although forecasting windows differ in structure and difficulty. We test whether whole Transformer blocks can be removed at inference time in two pretrained models: Chronos-Bolt and TimesFM 2.5. Our bypasses avoid the original block computation and preserve the dense model exactly when all blocks are enabled. Masks are selected only on calibration windows and evaluated on 1,088 held-out windows from ETTh1, Traffic, Weather, and Exchange. At moderate sparsity, Chronos-Bolt with 7/11 candidate blocks and TimesFM with 16/20 blocks show little change in mean relative forecasting quality across datasets while reducing median A100 batch-1 latency by 11.2% and 13.4%. More aggressive TimesFM skipping accelerates inference further but substantially harms Traffic and Weather. Across both models, different equal-budget masks win on different windows, and a retrospective oracle significantly improves over the best static mask on every dataset. However, dataset-specific masks transfer inconsistently, and Chronos representations do not support reliable unseen-domain routing. These results establish cross-architecture structured redundancy and input-dependent opportunity, but not a deployable dynamic inference method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 14 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

14 of 14 references.

References · 14

  1. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  2. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  3. 2024Chronos: Learning the Language of Time Seriescited by 759
  4. 2023A decoder-only foundation model for time-series forecastingcited by 692
  5. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  6. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  7. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  8. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  9. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
  10. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  11. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 251
  12. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
2 more
  1. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  2. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 113

Built on by

Not cited yet.