# High-Dimensional Analysis of Bootstrap Ensemble Classifiers

arXiv: https://arxiv.org/abs/2505.14587 · HTML: https://mlsanity.com/paper/2505.14587/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 36 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 7 of 7)

- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures](https://mlsanity.com/paper/2001.08370/) (2020)
- [The Implicit Regularization of Ordinary Least Squares Ensembles](https://mlsanity.com/paper/1910.04743/) (2019)
- [Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension](https://mlsanity.com/paper/2201.13383/) (2022)
- [Large Dimensional Analysis and Improvement of Multi Task Learning](https://mlsanity.com/paper/2009.01591/) (2020)
- [A replica analysis of under-bagging](https://mlsanity.com/paper/2404.09779/) (2024)
- [Precise Asymptotics of Bagging Regularized M-estimators](https://mlsanity.com/paper/2409.15252/) (2024)

## Built on by (most cited first, 3 of 3)

- [Incorporating priors in learning: a random matrix study under a teacher-student framework](https://mlsanity.com/paper/2509.22124/) (2025)
- [Concept activation vectors: a unifying view and adversarial attacks](https://mlsanity.com/paper/2509.22755/) (2025)
- [α-TCAV: A Unified Framework for Testing with Concept Activation Vectors](https://mlsanity.com/paper/2605.15688/) (2026)

## Abstract (arXiv)

Bootstrap methods have long been the cornerstone of ensemble learning in machine learning. This paper presents a theoretical analysis of bootstrap techniques applied to the Least Square Support Vector Machine (LSSVM) ensemble in the context of large and growing sample sizes and feature dimensionalities. Using tools from Random Matrix Theory, we investigate the performance of this classifier that aggregates decision functions from multiple weak classifiers, each trained on different subsets of the data. We provide insights into the use of bootstrap methods in high-dimensional settings, enhancing our understanding of their impact. Based on these findings, we propose strategies to select the number of subsets and the regularization parameter that maximize the performance of the LSSVM. Empirical experiments on synthetic and real-world datasets validate our theoretical results.