# Multi-Narrow Transformation as a Single-Model Ensemble: Boundary Conditions, Mechanisms, and Failure Modes

arXiv: https://arxiv.org/abs/2605.11530 · HTML: https://mlsanity.com/paper/2605.11530/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Don't Decay the Learning Rate, Increase the Batch Size](https://mlsanity.com/paper/1711.00489/) (2017)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [A Unified Theory of Diversity in Ensemble Learning](https://mlsanity.com/paper/2301.03962/) (2023)
- [Peer Collaborative Learning for Online Knowledge Distillation](https://mlsanity.com/paper/2006.04147/) (2020)
- [Packed-Ensembles for Efficient Uncertainty Estimation](https://mlsanity.com/paper/2210.09184/) (2022)
- [Joint Training of Deep Ensembles Fails Due to Learner Collusion](https://mlsanity.com/paper/2301.11323/) (2023)
- [Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training](https://mlsanity.com/paper/2011.14660/) (2020)
- [Coupled Ensembles of Neural Networks](https://mlsanity.com/paper/1709.06053/) (2017)
- [Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet](https://mlsanity.com/paper/2007.00649/) (2020)
- [Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones?](https://mlsanity.com/paper/2005.07292/) (2020)
- [Easy Ensemble: Simple Deep Ensemble Learning for Sensor-Based Human Activity Recognition](https://mlsanity.com/paper/2203.04153/) (2022)
- [Deep Negative Correlation Classification](https://mlsanity.com/paper/2212.07070/) (2022)

## Abstract (arXiv)

Single-model ensembles (SMEs) have attracted attention as a way to approximate some of the benefits of deep ensembles within a single network. However, under an approximately matched parameter budget, it remains unclear whether model capacity should be concentrated in a single wide pathway or redistributed into many narrow and independent members. We investigate this question through the Multi-Narrow (MN) transformation, which converts a baseline CNN into an SME of narrow, path-wise independent branches while approximately preserving the dominant parameter budget. We systematically compare Single-Wide and Multi-Narrow configurations across different training-data regimes, architectures, and datasets. The results show that the effectiveness of MN is strongly data-dependent: weakly partitioned or baseline-wide models are preferable in data-rich settings, whereas highly partitioned MN models consistently outperform the baseline in low-data settings. This tendency is reproduced across multiple CNN architectures and image-classification datasets, suggesting that it is not specific to a single benchmark or model family. Analysis of internal representations shows that high-MN models learn more diverse and less redundant path-wise features. In low-data regimes, this diversity is broadly utilized and improves generalization, whereas in data-rich regimes, training becomes imbalanced and prediction is dominated by a small subset of paths. These findings clarify when and why Multi-Narrow transformation is effective, and provide practical guidance for allocating model capacity between width and member multiplicity under a limited budget.