# A Margin-Maximizing Fine-Grained Ensemble Method

arXiv: https://arxiv.org/abs/2409.12849 · HTML: https://mlsanity.com/paper/2409.12849/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 19 Sep 2024.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://mlsanity.com/paper/1801.10578/) (2018)
- [Domain Adaptive Ensemble Learning](https://mlsanity.com/paper/2003.07325/) (2020)
- [Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection](https://mlsanity.com/paper/2103.04612/) (2021)
- [A Universal Approximation Result for Difference of log-sum-exp Neural Networks](https://mlsanity.com/paper/1905.08503/) (2019)
- [Evolutionary bagging for ensemble learning](https://mlsanity.com/paper/2208.02400/) (2022)
- [Doubly Stochastic Adaptive Neighbors Clustering via the Marcus Mapping](https://mlsanity.com/paper/2408.02932/) (2024)
- [Achieving More with Less: A Tensor-Optimization-Powered Ensemble Method](https://mlsanity.com/paper/2408.02936/) (2024)
- [Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule](https://mlsanity.com/paper/2309.09030/) (2023)

## Built on by (most cited first, 6 of 6)

- [Cooperation of Experts: Fusing Heterogeneous Information with Large Margin](https://mlsanity.com/paper/2505.20853/) (2025)
- [Riemannian Optimization on Relaxed Indicator Matrix Manifold](https://mlsanity.com/paper/2503.20505/) (2025)
- [Efficient and Effective Model Extraction](https://mlsanity.com/paper/2409.14122/) (2024)
- [Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers](https://mlsanity.com/paper/2602.05136/) (2026)
- [Boosting the Accuracy of Stock Market Prediction via Multi-Layer Hybrid MTL Structure](https://mlsanity.com/paper/2501.09760/) (2025)
- [Hadamard-Riemannian Optimization for Margin-Variance Ensemble](https://mlsanity.com/paper/2509.10189/) (2025)

## Abstract (arXiv)

Ensemble learning has achieved remarkable success in machine learning, but its reliance on numerous base learners limits its application in resource-constrained environments. This paper introduces an innovative "Margin-Maximizing Fine-Grained Ensemble Method" that achieves performance surpassing large-scale ensembles by meticulously optimizing a small number of learners and enhancing generalization capability. We propose a novel learnable confidence matrix, quantifying each classifier's confidence for each category, precisely capturing category-specific advantages of individual learners. Furthermore, we design a margin-based loss function, constructing a smooth and partially convex objective using the logsumexp technique. This approach improves optimization, eases convergence, and enables adaptive confidence allocation. Finally, we prove that the loss function is Lipschitz continuous, based on which we develop an efficient gradient optimization algorithm that simultaneously maximizes margins and dynamically adjusts learner weights. Extensive experiments demonstrate that our method outperforms traditional random forests using only one-tenth of the base learners and other state-of-the-art ensemble methods.