# Autoselection of the Ensemble of Convolutional Neural Networks with Second-Order Cone Programming

arXiv: https://arxiv.org/abs/2302.05950 · HTML: https://mlsanity.com/paper/2302.05950/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [The State of Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1902.09574/) (2019)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://mlsanity.com/paper/1705.07565/) (2017)
- [Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks](https://mlsanity.com/paper/1511.06314/) (2015)
- [Data-free parameter pruning for Deep Neural Networks](https://mlsanity.com/paper/1507.06149/) (2015)
- [Conditional Generation and Snapshot Learning in Neural Dialogue Systems](https://mlsanity.com/paper/1606.03352/) (2016)
- [Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation](https://mlsanity.com/paper/1606.04199/) (2016)
- [Efficient Adaptive Ensembling for Image Classification](https://mlsanity.com/paper/2206.07394/) (2022)
- [Convex Optimization on Functionals of Probability Densities](https://mlsanity.com/paper/2002.06488/) (2020)
- [FatNet: High Resolution Kernels for Classification Using Fully Convolutional Optical Neural Networks](https://mlsanity.com/paper/2210.16914/) (2022)
- [Second Order Cone Constrained Convex Relaxations for Nonconvex Quadratically Constrained Quadratic Programming](https://mlsanity.com/paper/1608.02096/) (2016)

## Abstract (arXiv)

Ensemble techniques are frequently encountered in machine learning and engineering problems since the method combines different models and produces an optimal predictive solution. The ensemble concept can be adapted to deep learning models to provide robustness and reliability. Due to the growth of the models in deep learning, using ensemble pruning is highly important to deal with computational complexity. Hence, this study proposes a mathematical model which prunes the ensemble of Convolutional Neural Networks (CNN) consisting of different depths and layers that maximizes accuracy and diversity simultaneously with a sparse second order conic optimization model. The proposed model is tested on CIFAR-10, CIFAR-100 and MNIST data sets which gives promising results while reducing the complexity of models, significantly.