mlsanityData to 7 Oct 2026

On the proliferation of support vectors in high dimensions

arXiv 2009.10670 ↗First posted 22 Sep 2020math.ST cs.LG stat.MLcited by 32

Not on the map. Why?

Abstract (arXiv)

The support vector machine (SVM) is a well-established classification method whose name refers to the particular training examples, called support vectors, that determine the maximum margin separating hyperplane. The SVM classifier is known to enjoy good generalization properties when the number of support vectors is small compared to the number of training examples. However, recent research has shown that in sufficiently high-dimensional linear classification problems, the SVM can generalize well despite a proliferation of support vectors where all training examples are support vectors. In this paper, we identify new deterministic equivalences for this phenomenon of support vector proliferation, and use them to (1) substantially broaden the conditions under which the phenomenon occurs in high-dimensional settings, and (2) prove a nearly matching converse result.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 8; 2023: 5; 2024: 2; 2025: 5; 2026: 2.

32 papers.

Sort:

  1. 2022Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Datacited by 74
  2. 2023Transformers as Support Vector Machinescited by 67
  3. 2021A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learningcited by 54
  4. 2021Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixturescited by 48
  5. 2021Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolationcited by 41
  6. 2023Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximizationcited by 29
  7. 2021The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networkscited by 18
  8. 2022Error Scaling Laws for Kernel Classification under Source and Capacity Conditionscited by 17
  9. 2021Harmless interpolation in regression and classification with structured featurescited by 14
  10. 2025Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regressioncited by 9
  11. 2021Support vector machines and linear regression coincide with very high-dimensional featurescited by 8
  12. 2022Generalization for multiclass classification with overparameterized linear modelscited by 8
20 more
  1. 2022Deep Linear Networks can Benignly Overfit when Shallow Ones Docited by 8
  2. 2025Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regressioncited by 7
  3. 2021The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularizationcited by 6
  4. 2021Minimax Supervised Clustering in the Anisotropic Gaussian Mixture Model: A new take on Robust Interpolationcited by 6
  5. 2022Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergencecited by 6
  6. 2025FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networkscited by 5
  7. 2021Depth Without the Magic: Inductive Bias of Natural Gradient Descentcited by 4
  8. 2022On how to avoid exacerbating spurious correlations when models are overparameterizedcited by 4
  9. 2023General Loss Functions Lead to (Approximate) Interpolation in High Dimensionscited by 4
  10. 2023New Equivalences Between Interpolation and SVMs: Kernels and Structured Featurescited by 3
  11. 2023Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Modelscited by 3
  12. 2024Optimal Downsampling for Imbalanced Classification with Generalized Linear Modelscited by 2
  13. 2026Large-Margin Hyperdimensional Computing: A Learning-Theoretical Perspectivecited by 1
  14. 2021Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspectivecited by 0
  15. 2022Interpolating Discriminant Functions in High-Dimensional Gaussian Latent Mixturescited by 0
  16. 2022Tight bounds for maximum ℓ1-margin classifierscited by 0
  17. 2024Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Netscited by 0
  18. 2025Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networkscited by 0
  19. 2025Variational Deep Learning via Implicit Regularizationcited by 0
  20. 2026Robust Federated Learning via Byzantine Filtering over Encrypted Updatescited by 0