# Statistical Mechanics of Support Vector Regression

arXiv: https://arxiv.org/abs/2412.05439 · HTML: https://mlsanity.com/paper/2412.05439/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 9 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 2 of 2)

- [Scaling and renormalization in high-dimensional regression](https://mlsanity.com/paper/2405.00592/) (2024)
- [A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimator](https://mlsanity.com/paper/2403.08938/) (2024)

## Abstract (arXiv)

A key problem in deep learning and computational neuroscience is relating the geometrical properties of neural representations to task performance. Here, we consider this problem for continuous decoding tasks where neural variability may affect task precision. Using methods from statistical mechanics, we study the average-case learning curves for varepsilon-insensitive Support Vector Regression (varepsilon-SVR) and discuss its capacity as a measure of linear decodability. Our analysis reveals a phase transition in training error at a critical load, capturing the interplay between the tolerance parameter varepsilon and neural variability. We uncover a double-descent phenomenon in the generalization error, showing that varepsilon acts as a regularizer, both suppressing and shifting these peaks. Theoretical predictions are validated both with toy models and deep neural networks, extending the theory of Support Vector Machines to continuous tasks with inherent neural variability.