# DivBO: Diversity-aware CASH for Ensemble Learning

arXiv: https://arxiv.org/abs/2302.03255 · HTML: https://mlsanity.com/paper/2302.03255/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 64 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [FLAML: A Fast and Lightweight AutoML Library](https://mlsanity.com/paper/1911.04706/) (2019)
- [Automated Machine Learning: From Principles to Practices](https://mlsanity.com/paper/1810.13306/) (2018)
- [Neural Ensemble Search for Uncertainty Estimation and Dataset Shift](https://mlsanity.com/paper/2006.08573/) (2020)
- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [An ADMM Based Framework for AutoML Pipeline Configuration](https://mlsanity.com/paper/1905.00424/) (2019)
- [OpenBox: A Generalized Black-box Optimization Service](https://mlsanity.com/paper/2106.00421/) (2021)
- [PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm](https://mlsanity.com/paper/2203.00638/) (2022)
- [Efficient Automatic CASH via Rising Bandits](https://mlsanity.com/paper/2012.04371/) (2020)
- [When does Diversity Help Generalization in Classification Ensembles?](https://mlsanity.com/paper/1910.13631/) (2019)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements](https://mlsanity.com/paper/2012.03011/) (2020)
- [Bayesian Hyperparameter Optimization for Ensemble Learning](https://mlsanity.com/paper/1605.06394/) (2016)
- [Efficient Diversity-Driven Ensemble for Deep Neural Networks](https://mlsanity.com/paper/2112.13316/) (2021)
- [Pre-train and Learn: Preserve Global Information for Graph Neural Networks](https://mlsanity.com/paper/1910.12241/) (2019)

## Built on by (most cited first, 6 of 6)

- [PSEO: Optimizing Post-hoc Stacking Ensemble Through Hyperparameter Tuning](https://mlsanity.com/paper/2508.05144/) (2025)
- [Rover: An online Spark SQL tuning service via generalized transfer learning](https://mlsanity.com/paper/2302.04046/) (2023)
- [Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH](https://mlsanity.com/paper/2601.12355/) (2026)
- [Regularized Neural Ensemblers](https://mlsanity.com/paper/2410.04520/) (2024)
- [HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data](https://mlsanity.com/paper/2603.10582/) (2026)
- [CoFEH: LLM-driven Feature Engineering Empowered by Collaborative Bayesian Hyperparameter Optimization](https://mlsanity.com/paper/2602.09851/) (2026)

## Abstract (arXiv)

The Combined Algorithm Selection and Hyperparameters optimization (CASH) problem is one of the fundamental problems in Automated Machine Learning (AutoML). Motivated by the success of ensemble learning, recent AutoML systems build post-hoc ensembles to output the final predictions instead of using the best single learner. However, while most CASH methods focus on searching for a single learner with the best performance, they neglect the diversity among base learners (i.e., they may suggest similar configurations to previously evaluated ones), which is also a crucial consideration when building an ensemble. To tackle this issue and further enhance the ensemble performance, we propose DivBO, a diversity-aware framework to inject explicit search of diversity into the CASH problems. In the framework, we propose to use a diversity surrogate to predict the pair-wise diversity of two unseen configurations. Furthermore, we introduce a temporary pool and a weighted acquisition function to guide the search of both performance and diversity based on Bayesian optimization. Empirical results on 15 public datasets show that DivBO achieves the best average ranks (1.82 and 1.73) on both validation and test errors among 10 compared methods, including post-hoc designs in recent AutoML systems and state-of-the-art baselines for ensemble learning on CASH problems.