# POCKET: Pruning Random Convolution Kernels for Time Series Classification from a Feature Selection Perspective

arXiv: https://arxiv.org/abs/2309.08499 · HTML: https://mlsanity.com/paper/2309.08499/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 71 references that are in the map.
- First posted on arXiv 15 Sep 2023.

## Builds on (most cited first, 27 of 27)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks](https://mlsanity.com/paper/1808.06866/) (2018)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Benchmarking Multivariate Time Series Classification Algorithms](https://mlsanity.com/paper/2007.13156/) (2020)
- [HIVE-COTE 2.0: a new meta ensemble for time series classification](https://mlsanity.com/paper/2104.07551/) (2021)
- [MultiRocket: Multiple pooling operators and transformations for fast and effective time series classification](https://mlsanity.com/paper/2102.00457/) (2021)
- [A Time Series Forest for Classification and Feature Extraction](https://mlsanity.com/paper/1302.2277/) (2013)
- [HYDRA: Competing convolutional kernels for fast and accurate time series classification](https://mlsanity.com/paper/2203.13652/) (2022)
- [The Canonical Interval Forest (CIF) Classifier for Time Series Classification](https://mlsanity.com/paper/2008.09172/) (2020)
- [Proximity Forest: An effective and scalable distance-based classifier for time series](https://mlsanity.com/paper/1808.10594/) (2018)
- [Improving Position Encoding of Transformers for Multivariate Time Series Classification](https://mlsanity.com/paper/2305.16642/) (2023)
- [The Temporal Dictionary Ensemble (TDE) Classifier for Time Series Classification](https://mlsanity.com/paper/2105.03841/) (2021)
- [Scalable Dictionary Classifiers for Time Series Classification](https://mlsanity.com/paper/1907.11815/) (2019)
- [HDC-MiniROCKET: Explicit Time Encoding in Time Series Classification with Hyperdimensional Computing](https://mlsanity.com/paper/2202.08055/) (2022)
- [Joint Matrix Decomposition for Deep Convolutional Neural Networks Compression](https://mlsanity.com/paper/2107.04386/) (2021)
- [S-Rocket: Selective Random Convolution Kernels for Time Series Classification](https://mlsanity.com/paper/2203.03445/) (2022)
- [WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks](https://mlsanity.com/paper/2302.08185/) (2023)
- [Taking ROCKET on an Efficiency Mission: Multivariate Time Series Classification with LightWaveS](https://mlsanity.com/paper/2204.01379/) (2022)
- [OPP-Miner: Order-preserving sequential pattern mining](https://mlsanity.com/paper/2202.03140/) (2022)
- [OPR-Miner: Order-preserving rule mining for time series](https://mlsanity.com/paper/2209.08932/) (2022)

## Built on by (most cited first, 5 of 5)

- [Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels](https://mlsanity.com/paper/2309.14518/) (2023)
- [Time series classification with random convolution kernels: pooling operators and input representations matter](https://mlsanity.com/paper/2409.01115/) (2024)
- [Breaking the accuracy-resource dilemma: a lightweight adaptive video inference enhancement](https://mlsanity.com/paper/2601.14568/) (2026)
- [Pruning Extensions and Efficiency Trade-Offs for Sustainable Time Series Classification](https://mlsanity.com/paper/2604.07953/) (2026)
- [Random Feature Pruning for Multi-Class Classification](https://mlsanity.com/paper/2610.15101/) (2026)

## Abstract (arXiv)

In recent years, two competitive time series classification models, namely, ROCKET and MINIROCKET, have garnered considerable attention due to their low training cost and high accuracy. However, they rely on a large number of random 1-D convolutional kernels to comprehensively capture features, which is incompatible with resource-constrained devices. Despite the development of heuristic algorithms designed to recognize and prune redundant kernels, the inherent time-consuming nature of evolutionary algorithms hinders efficient evaluation. To efficiently prune models, this paper eliminates feature groups contributing minimally to the classifier, thereby discarding the associated random kernels without direct evaluation. To this end, we incorporate both group-level (l2,1-norm) and element-level (l2-norm) regularizations to the classifier, formulating the pruning challenge as a group elastic net classification problem. An ADMM-based algorithm is initially introduced to solve the problem, but it is computationally intensive. Building on the ADMM-based algorithm, we then propose our core algorithm, POCKET, which significantly speeds up the process by dividing the task into two sequential stages. In Stage 1, POCKET utilizes dynamically varying penalties to efficiently achieve group sparsity within the classifier, removing features associated with zero weights and their corresponding kernels. In Stage 2, the remaining kernels and features are used to refit a l2-regularized classifier for enhanced performance. Experimental results on diverse time series datasets show that POCKET prunes up to 60% of kernels without a significant reduction in accuracy and performs 11× faster than its counterparts. Our code is publicly available at https://github.com/ShaowuChen/POCKET.