mlsanityData to 14 Oct 2026

TS-Fault: Benchmarking Time Series Forecasters Against Structural Faults

arXiv 2606.18539 ↗First posted 16 Jun 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Time series forecasting (TSF) underpins consequential decisions in energy, transportation, finance, and healthcare, yet TSF models are almost universally ranked by a single number (e.g., average error) on clean held-out data, under the implicit assumption that it predicts deployed reliability. However, real faults are not i.i.d noise but structured events with temporal shape, broken cross-variable dependencies, regime change coupled with missingness, and causal propagation across a sensing pipeline. Treating TSF robustness as a data-quality problem, we present TS-Fault, a benchmark that evaluates forecasting models under explicit, parameterized fault scenarios with controllable semantic difficulty. TS-Fault organizes recurring failures into four modes along two orthogonal axes (observation- vs mechanism-level; univariate vs multivariate) and injects each fault into the most prediction-critical window via a unified importance score. This design enables robustness to be tested against the structures models actually rely on, rather than reduced to generic noise sensitivity. We evaluate 21 models across 6 datasets, 4 modes, and 5 difficulty levels under a paired clean/corrupt protocol. The results reveal three findings that contradict common leaderboard intuition: (i) clean-data accuracy anti-correlates with robustness; (ii) clean rankings are preserved under observation-level faults but reshuffled under mechanism-level faults; and (iii) all catastrophic failures occur under mechanism-level faults, with foundation models achieving the highest clean-data accuracy yet exhibiting the greatest fragility. The code is publicly available at https://github.com/Ray-zyy/TS-Fault.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 52 of its 84 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

52 of 84 references.

References · 52

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  10. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  11. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  12. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
38 more
  1. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  2. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  3. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  4. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  5. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  6. 2024Chronos: Learning the Language of Time Seriescited by 759
  7. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  8. 2023A decoder-only foundation model for time-series forecastingcited by 692
  9. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  10. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  11. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  12. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  13. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  14. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  15. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 346
  16. 2018BRITS: Bidirectional Recurrent Imputation for Time Seriescited by 290
  17. 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 273
  18. 2021Monash Time Series Forecasting Archivecited by 267
  19. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 249
  20. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 248
  21. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 240
  22. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  23. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  24. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 177
  25. 2022SAITS: Self-Attention-based Imputation for Time Seriescited by 169
  26. 2018Categorizing Variants of Goodhart's Lawcited by 149
  27. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 148
  28. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 147
  29. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 117
  30. 2021On Interaction Between Augmentations and Corruptions in Natural Corruption Robustnesscited by 97
  31. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 90
  32. 2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 74
  33. 2025fev-bench: A Realistic Benchmark for Time Series Forecastingcited by 64
  34. 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 58
  35. 2023Multi-resolution Time-Series Transformer for Long-term Forecastingcited by 41
  36. 2023ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizonscited by 36
  37. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 30
  38. 2022Practical Adversarial Attacks on Spatiotemporal Traffic Forecasting Modelscited by 29

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.