# Surrogate assisted diversity estimation in neural ensemble search

arXiv: https://arxiv.org/abs/2607.26940 · HTML: https://mlsanity.com/paper/2607.26940/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 12 of 12)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search](https://mlsanity.com/paper/1910.11858/) (2019)
- [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019)
- [GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training](https://mlsanity.com/paper/2009.03294/) (2020)
- [Neural Ensemble Search for Uncertainty Estimation and Dataset Shift](https://mlsanity.com/paper/2006.08573/) (2020)
- [AdaNet: Adaptive Structural Learning of Artificial Neural Networks](https://mlsanity.com/paper/1607.01097/) (2016)
- [One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking](https://mlsanity.com/paper/2104.00597/) (2021)

## Abstract (arXiv)

Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of individual models. Most neural architecture search (NAS) methods are computationally expensive. Extending them to neural ensemble search (NES), which requires joint optimization of individual architectures and their ensemble composition, leads to an exponential growth of the search space and makes the problem computationally intractable. To address this, we introduce a dual-objective surrogate-guided ensemble search: candidate architectures are represented as directed acyclic graphs, and two surrogate models are trained independently to estimate predictive accuracy and diversity potential. Their combined estimates guide an NES framework that efficiently identifies architectures that are both individually strong and collectively diverse. Our final ensemble achieves competitive or superior performance compared to standard baselines such as Deep Ensembles and Random Search on FashionMNIST, CIFAR-10, and CIFAR-100.