# The Effect of Missingness-Pattern Mismatch on Method Selection for Time-Series Classification: A Controlled Empirical Study

arXiv: https://arxiv.org/abs/2610.05368 · HTML: https://mlsanity.com/paper/2610.05368/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Builds on (most cited first, 10 of 10)

- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Bake off redux: a review and experimental evaluation of recent time series classification algorithms](https://mlsanity.com/paper/2304.13029/) (2023)
- [aeon: a Python toolkit for learning from time series](https://mlsanity.com/paper/2406.14231/) (2024)
- [Highly comparative feature-based time-series classification](https://mlsanity.com/paper/1401.3531/) (2014)
- [Deep Imputation of Missing Values in Time Series Health Data: A Review with Benchmarking](https://mlsanity.com/paper/2302.10902/) (2023)
- [Probabilistic Imputation for Time-series Classification with Missing Data](https://mlsanity.com/paper/2308.06738/) (2023)
- [Domain Adaptation under Missingness Shift](https://mlsanity.com/paper/2211.02093/) (2022)
- [An Experimental Evaluation of Nearest Neighbour Time Series Classification](https://mlsanity.com/paper/1406.4757/) (2014)
- [Robust prediction under missingness shifts](https://mlsanity.com/paper/2406.16484/) (2024)

## Abstract (arXiv)

Classifiers for time-series classification are commonly selected on validation data, but the temporal pattern of missing observations at deployment may differ from the pattern seen during validation. We examine whether such a mismatch affects validation-based classifier selection. In a controlled 2 × 2 design, validation and test sets of 64 univariate UCR datasets were masked with either random point missingness or circular block missingness at six rates from 5% to 30%, imputed by linear interpolation, and used to select among three prespecified candidates: 1NN-DTW, MiniRocket with a Ridge classifier, and a statistical-feature Random Forest. Training data remained complete, and selections made under matched and mismatched validation patterns were compared on the same masked test sets. Mismatched validation reduced the test balanced accuracy of the selected classifier by 1.14 percentage points on average (95% CI 0.79 to 1.51), with losses on 49 of the 64 datasets. The loss was negligible at 5% missingness and increased to 2.46 percentage points at 30%. It was concentrated in point-masked deployment (1.84 percentage points), where block-masked validation shifted selection away from the usually best candidate, while the effect for block-masked deployment was small and not significant. Mismatch changed the selected classifier in 35.5% of paired comparisons, but a changed selection did not always reduce performance. A supplementary analysis with non-wrapping linear blocks reproduced these findings with a larger effect (1.67 percentage points). Matching the missingness pattern of validation data to the expected deployment pattern is therefore a simple safeguard for method selection, particularly at higher missingness rates.