# Utilizing Data Fingerprints for Privacy-Preserving Algorithm Selection in Time Series Classification: Performance and Uncertainty Estimation on Unseen Datasets

arXiv: https://arxiv.org/abs/2409.08636 · HTML: https://mlsanity.com/paper/2409.08636/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 66 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 29 of 29)

- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020)
- [Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution](https://mlsanity.com/paper/1804.09081/) (2018)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [HIVE-COTE 2.0: a new meta ensemble for time series classification](https://mlsanity.com/paper/2104.07551/) (2021)
- [Bake off redux: a review and experimental evaluation of recent time series classification algorithms](https://mlsanity.com/paper/2304.13029/) (2023)
- [MultiRocket: Multiple pooling operators and transformations for fast and effective time series classification](https://mlsanity.com/paper/2102.00457/) (2021)
- [A Time Series Forest for Classification and Feature Extraction](https://mlsanity.com/paper/1302.2277/) (2013)
- [TS-CHIEF: A Scalable and Accurate Forest Algorithm for Time Series Classification](https://mlsanity.com/paper/1906.10329/) (2019)
- [HYDRA: Competing convolutional kernels for fast and accurate time series classification](https://mlsanity.com/paper/2203.13652/) (2022)
- [The Canonical Interval Forest (CIF) Classifier for Time Series Classification](https://mlsanity.com/paper/2008.09172/) (2020)
- [Fast and Accurate Time Series Classification with WEASEL](https://mlsanity.com/paper/1701.07681/) (2017)
- [Proximity Forest: An effective and scalable distance-based classifier for time series](https://mlsanity.com/paper/1808.10594/) (2018)
- [ASlib: A Benchmark Library for Algorithm Selection](https://mlsanity.com/paper/1506.02465/) (2015)
- [shapeDTW: shape Dynamic Time Warping](https://mlsanity.com/paper/1606.01601/) (2016)
- [A Survey of Methods for Automated Algorithm Configuration](https://mlsanity.com/paper/2202.01651/) (2022)
- [Random Dilated Shapelet Transform: A New Approach for Time Series Shapelets](https://mlsanity.com/paper/2109.13514/) (2021)
- [Scalable Dictionary Classifiers for Time Series Classification](https://mlsanity.com/paper/1907.11815/) (2019)
- [The FreshPRINCE: A Simple Transformation Based Pipeline Time Series Classifier](https://mlsanity.com/paper/2201.12048/) (2022)
- [Applications of shapelet transform to time series classification of earthquake, wind and wave data](https://mlsanity.com/paper/2004.11243/) (2020)
- [Fast, Accurate and Interpretable Time Series Classification Through Randomization](https://mlsanity.com/paper/2105.14876/) (2021)
- [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://mlsanity.com/paper/2203.08399/) (2022)
- [Towards Privacy-Preserving Neural Architecture Search](https://mlsanity.com/paper/2204.10958/) (2022)

## Abstract (arXiv)

The selection of algorithms is a crucial step in designing AI services for real-world time series classification use cases. Traditional methods such as neural architecture search, automated machine learning, combined algorithm selection, and hyperparameter optimizations are effective but require considerable computational resources and necessitate access to all data points to run their optimizations. In this work, we introduce a novel data fingerprint that describes any time series classification dataset in a privacy-preserving manner and provides insight into the algorithm selection problem without requiring training on the (unseen) dataset. By decomposing the multi-target regression problem, only our data fingerprints are used to estimate algorithm performance and uncertainty in a scalable and adaptable manner. Our approach is evaluated on the 112 University of California riverside benchmark datasets, demonstrating its effectiveness in predicting the performance of 35 state-of-the-art algorithms and providing valuable insights for effective algorithm selection in time series classification service systems, improving a naive baseline by 7.32% on average in estimating the mean performance and 15.81% in estimating the uncertainty.