# Fast and interpretable Support Vector Classification based on the truncated ANOVA decomposition

arXiv: https://arxiv.org/abs/2402.02438 · HTML: https://mlsanity.com/paper/2402.02438/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 59 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Builds on (most cited first, 4 of 4)

- [Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection](https://mlsanity.com/paper/2209.09326/) (2022)
- [Interpretable Approximation of High-Dimensional Data](https://mlsanity.com/paper/2103.13787/) (2021)
- [Learning in High-Dimensional Feature Spaces Using ANOVA-Based Fast Matrix-Vector Multiplication](https://mlsanity.com/paper/2111.10140/) (2021)
- [A Preconditioned Interior Point Method for Support Vector Machines Using an ANOVA-Decomposition and NFFT-Based Matrix-Vector Products](https://mlsanity.com/paper/2312.00538/) (2023)

## Abstract (arXiv)

Support Vector Machines (SVMs) are an important tool for performing classification on scattered data, where one usually has to deal with many data points in high-dimensional spaces. We propose solving SVMs in primal form using feature maps based on trigonometric functions or wavelets. In small dimensional settings the Fast Fourier Transform (FFT) and related methods are a powerful tool in order to deal with the considered basis functions. For growing dimensions the classical FFT-based methods become inefficient due to the curse of dimensionality. Therefore, we restrict ourselves to multivariate basis functions, each of which only depends on a small number of dimensions. This is motivated by the well-known sparsity of effects and recent results regarding the reconstruction of functions from scattered data in terms of truncated analysis of variance (ANOVA) decompositions, which makes the resulting model even interpretable in terms of importance of the features as well as their couplings. The usage of small superposition dimensions has the consequence that the computational effort no longer grows exponentially but only polynomially with respect to the dimension. In order to enforce sparsity regarding the basis coefficients, we use the frequently applied ℓ2-norm and, in addition, ℓ1-norm regularization. The found classifying function, which is the linear combination of basis functions, and its variance can then be analyzed in terms of the classical ANOVA decomposition of functions. Based on numerical examples we show that we are able to recover the signum of a function that perfectly fits our model assumptions. Furthermore, we perform classification on different artificial and real-world data sets. We obtain better results with ℓ1-norm regularization, both in terms of accuracy and clarity of interpretability.