# Random Feature Pruning for Multi-Class Classification

arXiv: https://arxiv.org/abs/2610.15101 · HTML: https://mlsanity.com/paper/2610.15101/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 10 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions](https://mlsanity.com/paper/1502.06800/) (2015)
- [Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond](https://mlsanity.com/paper/2004.11154/) (2020)
- [MultiRocket: Multiple pooling operators and transformations for fast and effective time series classification](https://mlsanity.com/paper/2102.00457/) (2021)
- [HYDRA: Competing convolutional kernels for fast and accurate time series classification](https://mlsanity.com/paper/2203.13652/) (2022)
- [On the Spectrum of Random Features Maps of High Dimensional Data](https://mlsanity.com/paper/1805.11916/) (2018)
- [HDC-MiniROCKET: Explicit Time Encoding in Time Series Classification with Hyperdimensional Computing](https://mlsanity.com/paper/2202.08055/) (2022)
- [Concentration of Random Feature Matrices in High-Dimensions](https://mlsanity.com/paper/2204.06935/) (2022)
- [Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels](https://mlsanity.com/paper/2309.14518/) (2023)
- [S-Rocket: Selective Random Convolution Kernels for Time Series Classification](https://mlsanity.com/paper/2203.03445/) (2022)
- [Overparameterized random feature regression with nearly orthogonal data](https://mlsanity.com/paper/2211.06077/) (2022)
- [POCKET: Pruning Random Convolution Kernels for Time Series Classification from a Feature Selection Perspective](https://mlsanity.com/paper/2309.08499/) (2023)
- [Regularized Random Fourier Features and Finite Element Reconstruction for Operator Learning in Sobolev Space](https://mlsanity.com/paper/2512.17884/) (2025)
- [Scalable Random Wavelet Features: Efficient Non-Stationary Kernel Approximation with Convergence Guarantees](https://mlsanity.com/paper/2602.00987/) (2026)
- [Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)](https://mlsanity.com/paper/1401.6240/) (2014)
- [Multiple Output Regression with Latent Noise](https://mlsanity.com/paper/1410.7365/) (2014)
- [Model Selection Through Model Sorting](https://mlsanity.com/paper/2409.09674/) (2024)
- [HIT-ROCKET: Hadamard-vector Inner-product Transformer for ROCKET](https://mlsanity.com/paper/2511.01572/) (2025)
- [Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators](https://mlsanity.com/paper/2603.00971/) (2026)

## Abstract (arXiv)

Weighted sums of random kitchen sinks (W-RKS) is a commonly used feature extraction technique for supervised learning tasks. A major drawback of this method is the redundancy and ineffectiveness of many extracted random features. This paper investigates efficient methods for reducing such redundancy in multi-class classification problems. We specifically formulate feature selection as a variant of the sequentially nested model selection problem and introduce the Successive Empirical Excess Risk Difference Maximization (SEED-MAX) algorithm. It recursively computes empirical risks to identify the most parsimonious yet informative set of features. We derive an analytical lower bound for the probability of correct feature selection by the SEED-MAX algorithm. This lower bound on the probability of correct feature selection can be made arbitrarily close to one by increasing the training dataset size. To further enhance the rate of pruning, we evaluate the features retained by SEED-MAX using the Fisher test (F-Test). Empirical evaluations on UCR time-series classification datasets show that the sequential application of SEED-MAX with the Fisher test achieves approximately 92 percent rate of pruning with negligible degradation in classification accuracy. Implementations of the SEED-MAX and SEED-MAX-F-Test methods are available at https://github.com/OmidiAmin/seed-max-ftest.