# A hybrid ensemble method with negative correlation learning for regression

arXiv: https://arxiv.org/abs/2104.02317 · HTML: https://mlsanity.com/paper/2104.02317/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 81 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 4 of 4)

- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [Model Selection Techniques -- An Overview](https://mlsanity.com/paper/1810.09583/) (2018)
- [When does Diversity Help Generalization in Classification Ensembles?](https://mlsanity.com/paper/1910.13631/) (2019)
- [Diversity and degrees of freedom in regression ensembles](https://mlsanity.com/paper/1803.00314/) (2018)

## Abstract (arXiv)

Hybrid ensemble, an essential branch of ensembles, has flourished in the regression field, with studies confirming diversity's importance. However, previous ensembles consider diversity in the sub-model training stage, with limited improvement compared to single models. In contrast, this study automatically selects and weights sub-models from a heterogeneous model pool. It solves an optimization problem using an interior-point filtering linear-search algorithm. The objective function innovatively incorporates negative correlation learning as a penalty term, with which a diverse model subset can be selected. The best sub-models from each model class are selected to build the NCL ensemble, which performance is better than the simple average and other state-of-the-art weighting methods. It is also possible to improve the NCL ensemble with a regularization term in the objective function. In practice, it is difficult to conclude the optimal sub-model for a dataset prior due to the model uncertainty. Regardless, our method would achieve comparable accuracy as the potential optimal sub-models. In conclusion, the value of this study lies in its ease of use and effectiveness, allowing the hybrid ensemble to embrace diversity and accuracy.