# Generalized Negative Correlation Learning for Deep Ensembling

arXiv: https://arxiv.org/abs/2011.02952 · HTML: https://mlsanity.com/paper/2011.02952/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2020.

## Built on by (most cited first, 17 of 17)

- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [A Unified Theory of Diversity in Ensemble Learning](https://mlsanity.com/paper/2301.03962/) (2023)
- [Fusing Models with Complementary Expertise](https://mlsanity.com/paper/2310.01542/) (2023)
- [Diversity and Generalization in Neural Network Ensembles](https://mlsanity.com/paper/2110.13786/) (2021)
- [Joint Training of Deep Ensembles Fails Due to Learner Collusion](https://mlsanity.com/paper/2301.11323/) (2023)
- [Pathologies of Predictive Diversity in Deep Ensembles](https://mlsanity.com/paper/2302.00704/) (2023)
- [Leveraging Ensemble Diversity for Robust Self-Training in the Presence of Sample Selection Bias](https://mlsanity.com/paper/2310.14814/) (2023)
- [Semi-Supervised Deep Ensembles for Blind Image Quality Assessment](https://mlsanity.com/paper/2106.14008/) (2021)
- [Ensembling over Classifiers: a Bias-Variance Perspective](https://mlsanity.com/paper/2206.10566/) (2022)
- [There is no Double-Descent in Random Forests](https://mlsanity.com/paper/2111.04409/) (2021)
- [Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement](https://mlsanity.com/paper/2110.10075/) (2021)
- [Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis](https://mlsanity.com/paper/2209.00439/) (2022)
- [Exploring Resiliency to Natural Image Corruptions in Deep Learning using Design Diversity](https://mlsanity.com/paper/2303.09283/) (2023)
- [On Uniform, Bayesian, and PAC-Bayesian Deep Ensembles](https://mlsanity.com/paper/2406.05469/) (2024)
- [Decoupled Complementary Spectral-Spatial Learning for Background Representation Enhancement in Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2510.18781/) (2025)
- [From the Loss Landscape to Diverse Feature Learning in Neural Networks](https://mlsanity.com/paper/2608.28948/) (2026)
- [Virtual neural networks: hundreds of souls in a body](https://mlsanity.com/paper/2609.24782/) (2026)

## Abstract (arXiv)

Ensemble algorithms offer state of the art performance in many machine learning applications. A common explanation for their excellent performance is due to the bias-variance decomposition of the mean squared error which shows that the algorithm's error can be decomposed into its bias and variance. Both quantities are often opposed to each other and ensembles offer an effective way to manage them as they reduce the variance through a diverse set of base learners while keeping the bias low at the same time. Even though there have been numerous works on decomposing other loss functions, the exact mathematical connection is rarely exploited explicitly for ensembling, but merely used as a guiding principle. In this paper, we formulate a generalized bias-variance decomposition for arbitrary twice differentiable loss functions and study it in the context of Deep Learning. We use this decomposition to derive a Generalized Negative Correlation Learning (GNCL) algorithm which offers explicit control over the ensemble's diversity and smoothly interpolates between the two extremes of independent training and the joint training of the ensemble. We show how GNCL encapsulates many previous works and discuss under which circumstances training of an ensemble of Neural Networks might fail and what ensembling method should be favored depending on the choice of the individual networks. We make our code publicly available under https://github.com/sbuschjaeger/gncl