# Eliminating Multicollinearity Issues in Neural Network Ensembles: Incremental, Negatively Correlated, Optimal Convex Blending

arXiv: https://arxiv.org/abs/2104.14715 · HTML: https://mlsanity.com/paper/2104.14715/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 26 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 2 of 2)

- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Diversity and degrees of freedom in regression ensembles](https://mlsanity.com/paper/1803.00314/) (2018)

## Abstract (arXiv)

Given a {features, target} dataset, we introduce an incremental algorithm that constructs an aggregate regressor, using an ensemble of neural networks. It is well known that ensemble methods suffer from the multicollinearity issue, which is the manifestation of redundancy arising mainly due to the common training-dataset. In the present incremental approach, at each stage we optimally blend the aggregate regressor with a newly trained neural network under a convexity constraint which, if necessary, induces negative correlations. Under this framework, collinearity issues do not arise at all, rendering so the method both accurate and robust.