# Statistical comparisons of time-series feature sets on classification tasks

arXiv: https://arxiv.org/abs/2608.01586 · HTML: https://mlsanity.com/paper/2608.01586/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 69 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [Benchmarking Multivariate Time Series Classification Algorithms](https://mlsanity.com/paper/2007.13156/) (2020)
- [catch22: CAnonical Time-series CHaracteristics](https://mlsanity.com/paper/1901.10200/) (2019)
- [Bake off redux: a review and experimental evaluation of recent time series classification algorithms](https://mlsanity.com/paper/2304.13029/) (2023)
- [Scientific Machine Learning Benchmarks](https://mlsanity.com/paper/2110.12773/) (2021)
- [Highly comparative feature-based time-series classification](https://mlsanity.com/paper/1401.3531/) (2014)
- [Highly comparative time-series analysis: The empirical structure of time series and their methods](https://mlsanity.com/paper/1304.1209/) (2013)
- [Distributed and parallel time series feature extraction for industrial big data applications](https://mlsanity.com/paper/1610.07717/) (2016)
- [An Approach to Multiple Comparison Benchmark Evaluations that is Stable Under Manipulation of the Comparate Set](https://mlsanity.com/paper/2305.11921/) (2023)
- [Feature-based time-series analysis](https://mlsanity.com/paper/1709.08055/) (2017)
- [Ridge Regularization: an Essential Concept in Data Science](https://mlsanity.com/paper/2006.00371/) (2020)
- [Unsupervised Feature Based Algorithms for Time Series Extrinsic Regression](https://mlsanity.com/paper/2305.01429/) (2023)
- [Applications of shapelet transform to time series classification of earthquake, wind and wave data](https://mlsanity.com/paper/2004.11243/) (2020)
- [An Empirical Evaluation of Time-Series Feature Sets](https://mlsanity.com/paper/2110.10914/) (2021)
- [Never a Dull Moment: Distributional Properties as a Baseline for Time-Series Classification](https://mlsanity.com/paper/2303.17809/) (2023)

## Abstract (arXiv)

In recent years, numerous open-source software libraries have been developed for computing sets of features from univariate time series. The type and number of features vary across these feature sets, which have been constructed with varying disciplinary perspectives on quantifying structure in time-series data. To date, the relative strengths and weaknesses of these feature sets on time-series classification problems remains largely unexplored. Here we aimed to understand the relative performance of six open-source feature sets and three baseline feature sets (based on distributional and/or basic spectral structure) across 124 univariate time-series classification problems using a normalization-based approach to problem-level benchmarking that better indexes the relative strengths and weaknesses of different algorithms compared to prior rank-based approaches. Despite their dramatic differences in size, composition, and computation time, we found that feature sets performed relatively similarly overall (85.3% of pairwise comparisons resulted in ties), with the largest feature set, tsfresh, exhibiting the strongest overall performance (29.03% wins across all pairwise comparisons against other feature sets). We also highlighted specific problems on which the specific composition of a given feature set gave it a substantial performance advantage or disadvantage, and problems where simple baselines comprised of Fourier coefficients and quantiles were sufficient to achieve strong performance. Our results demonstrate the need to consider problem-level performance when benchmarking time-series feature sets, and highlight the importance of feature make-up in driving relative classification performance.