# Exploring Model Learning Heterogeneity for Boosting Ensemble Robustness

arXiv: https://arxiv.org/abs/2310.02237 · HTML: https://mlsanity.com/paper/2310.02237/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution](https://mlsanity.com/paper/2006.02334/) (2020)
- [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://mlsanity.com/paper/1901.08846/) (2019)
- [Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving](https://mlsanity.com/paper/1907.06826/) (2019)
- [Transferable Adversarial Attacks for Image and Video Object Detection](https://mlsanity.com/paper/1811.12641/) (2018)
- [Robust Learning Through Cross-Task Consistency](https://mlsanity.com/paper/2006.04096/) (2020)
- [Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environments](https://mlsanity.com/paper/1707.05733/) (2017)
- [Robust Adversarial Perturbation on Deep Proposal-based Models](https://mlsanity.com/paper/1809.05962/) (2018)
- [Robustness via Cross-Domain Ensembles](https://mlsanity.com/paper/2103.10919/) (2021)
- [Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness](https://mlsanity.com/paper/1908.11091/) (2019)
- [Understanding Object Detection Through An Adversarial Lens](https://mlsanity.com/paper/2007.05828/) (2020)
- [Robust Deep Learning Ensemble against Deception](https://mlsanity.com/paper/2009.06589/) (2020)
- [Cross-Layer Strategic Ensemble Defense Against Adversarial Examples](https://mlsanity.com/paper/1910.01742/) (2019)
- [Denoising and Verification Cross-Layer Ensemble Against Black-box Adversarial Attacks](https://mlsanity.com/paper/1908.07667/) (2019)
- [Promoting High Diversity Ensemble Learning with EnsembleBench](https://mlsanity.com/paper/2010.10623/) (2020)

## Built on by (most cited first, 6 of 6)

- [Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance](https://mlsanity.com/paper/2402.01096/) (2024)
- [Hierarchical Pruning of Deep Ensembles with Focal Diversity](https://mlsanity.com/paper/2311.10293/) (2023)
- [How Effective are Large Time Series Models in Hydrology? A Study on Water Level Forecasting in Everglades](https://mlsanity.com/paper/2505.01415/) (2025)
- [Accurate and Efficient Two-Stage Gun Detection in Video](https://mlsanity.com/paper/2503.06317/) (2025)
- [A Diversity-optimized Deep Ensemble Approach for Accurate Plant Leaf Disease Detection](https://mlsanity.com/paper/2511.16982/) (2025)
- [LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detection](https://mlsanity.com/paper/2607.06592/) (2026)

## Abstract (arXiv)

Deep neural network ensembles hold the potential of improving generalization performance for complex learning tasks. This paper presents formal analysis and empirical evaluation to show that heterogeneous deep ensembles with high ensemble diversity can effectively leverage model learning heterogeneity to boost ensemble robustness. We first show that heterogeneous DNN models trained for solving the same learning problem, e.g., object detection, can significantly strengthen the mean average precision (mAP) through our weighted bounding box ensemble consensus method. Second, we further compose ensembles of heterogeneous models for solving different learning problems, e.g., object detection and semantic segmentation, by introducing the connected component labeling (CCL) based alignment. We show that this two-tier heterogeneity driven ensemble construction method can compose an ensemble team that promotes high ensemble diversity and low negative correlation among member models of the ensemble, strengthening ensemble robustness against both negative examples and adversarial attacks. Third, we provide a formal analysis of the ensemble robustness in terms of negative correlation. Extensive experiments validate the enhanced robustness of heterogeneous ensembles in both benign and adversarial settings. The source codes are available on GitHub at https://github.com/git-disl/HeteRobust.