# Less Is More: A Comprehensive Framework for the Number of Components of Ensemble Classifiers

arXiv: https://arxiv.org/abs/1709.02925 · HTML: https://mlsanity.com/paper/1709.02925/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2017.

## Built on by (most cited first, 17 of 17)

- [Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2410.15021/) (2024)
- [Bag of Tricks for In-Distribution Calibration of Pretrained Transformers](https://mlsanity.com/paper/2302.06690/) (2023)
- [On-the-Fly Ensemble Pruning in Evolving Data Streams](https://mlsanity.com/paper/2109.07611/) (2021)
- [A Broad Ensemble Learning System for Drifting Stream Classification](https://mlsanity.com/paper/2110.03540/) (2021)
- [DynED: Dynamic Ensemble Diversification in Data Stream Classification](https://mlsanity.com/paper/2308.10807/) (2023)
- [Selecting a number of voters for a voting ensemble](https://mlsanity.com/paper/2104.11833/) (2021)
- [A Dataset-Level Geometric Framework for Ensemble Classifiers](https://mlsanity.com/paper/2106.08658/) (2021)
- [Meta-Ensemble Parameter Learning](https://mlsanity.com/paper/2210.01973/) (2022)
- [Neural Diversity Regularizes Hallucinations in Language Models](https://mlsanity.com/paper/2510.20690/) (2025)
- [EnHDC: Ensemble Learning for Brain-Inspired Hyperdimensional Computing](https://mlsanity.com/paper/2203.13542/) (2022)
- [Faster, Lighter, More Accurate: A Deep Learning Ensemble for Content Moderation](https://mlsanity.com/paper/2309.05150/) (2023)
- [The Case for High-Accuracy Classification: Think Small, Think Many!](https://mlsanity.com/paper/2103.10350/) (2021)
- [A Multi-View Learning Approach to Enhance Automatic 12-Lead ECG Diagnosis Performance](https://mlsanity.com/paper/2208.00323/) (2022)
- [Leveraging Linear Independence of Component Classifiers: Optimizing Size and Prediction Accuracy for Online Ensembles](https://mlsanity.com/paper/2308.14175/) (2023)
- [Advancing sleep detection by modelling weak label sets: A novel weakly supervised learning approach](https://mlsanity.com/paper/2402.17601/) (2024)
- [Logifold: A Geometrical Foundation of Ensemble Machine Learning](https://mlsanity.com/paper/2407.16177/) (2024)
- [Ensemble Performance Through the Lens of Linear Independence of Classifier Votes in Data Streams](https://mlsanity.com/paper/2511.21465/) (2025)

## Abstract (arXiv)

The number of component classifiers chosen for an ensemble greatly impacts the prediction ability. In this paper, we use a geometric framework for a priori determining the ensemble size, which is applicable to most of existing batch and online ensemble classifiers. There are only a limited number of studies on the ensemble size examining Majority Voting (MV) and Weighted Majority Voting (WMV). Almost all of them are designed for batch-mode, hardly addressing online environments. Big data dimensions and resource limitations, in terms of time and memory, make determination of ensemble size crucial, especially for online environments. For the MV aggregation rule, our framework proves that the more strong components we add to the ensemble, the more accurate predictions we can achieve. For the WMV aggregation rule, our framework proves the existence of an ideal number of components, which is equal to the number of class labels, with the premise that components are completely independent of each other and strong enough. While giving the exact definition for a strong and independent classifier in the context of an ensemble is a challenging task, our proposed geometric framework provides a theoretical explanation of diversity and its impact on the accuracy of predictions. We conduct a series of experimental evaluations to show the practical value of our theorems and existing challenges.