# Temporal cross-validation impacts multivariate time series subsequence anomaly detection evaluation

arXiv: https://arxiv.org/abs/2506.12183 · HTML: https://mlsanity.com/paper/2506.12183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 85 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [A review on outlier/anomaly detection in time series data](https://mlsanity.com/paper/2002.04236/) (2020)
- [Multivariate Time-series Anomaly Detection via Graph Attention Network](https://mlsanity.com/paper/2009.02040/) (2020)
- [Benchmarking Multivariate Time Series Classification Algorithms](https://mlsanity.com/paper/2007.13156/) (2020)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [Evaluating time series forecasting models: An empirical study on performance estimation methods](https://mlsanity.com/paper/1905.11744/) (2019)
- [A Comprehensive Guide to CAN IDS Data & Introduction of the ROAD Dataset](https://mlsanity.com/paper/2012.14600/) (2020)
- [CAN-D: A Modular Four-Step Pipeline for Comprehensively Decoding Controller Area Network Data](https://mlsanity.com/paper/2006.05993/) (2020)
- [Detecting CAN Masquerade Attacks with Signal Clustering Similarity](https://mlsanity.com/paper/2201.02665/) (2022)
- [IB-GAN: A Unified Approach for Multivariate Time Series Classification under Class Imbalance](https://mlsanity.com/paper/2110.07460/) (2021)
- [Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN](https://mlsanity.com/paper/2406.13778/) (2024)
- [Bad practices in evaluation methodology relevant to class-imbalanced problems](https://mlsanity.com/paper/1812.01388/) (2018)

## Abstract (arXiv)

Evaluating anomaly detection in multivariate time series (MTS) requires careful consideration of temporal dependencies, particularly when detecting subsequence anomalies common in fault detection scenarios. While time series cross-validation (TSCV) techniques aim to preserve temporal ordering during model evaluation, their impact on classifier performance remains underexplored. This study systematically investigates the effect of TSCV strategy on the precision-recall characteristics of classifiers trained to detect fault-like anomalies in MTS datasets. We compare walk-forward (WF) and sliding window (SW) methods across a range of validation partition configurations and classifier types, including shallow learners and deep learning (DL) classifiers. Results show that SW consistently yields higher median AUC-PR scores and reduced fold-to-fold performance variance, particularly for deep architectures sensitive to localized temporal continuity. Furthermore, we find that classifier generalization is sensitive to the number and structure of temporal partitions, with overlapping windows preserving fault signatures more effectively at lower fold counts. A classifier-level stratified analysis reveals that certain algorithms, such as random forests (RF), maintain stable performance across validation schemes, whereas others exhibit marked sensitivity. This study demonstrates that TSCV design in benchmarking anomaly detection models on streaming time series and provide guidance for selecting evaluation strategies in temporally structured learning environments.