# Any-Width Networks

arXiv: https://arxiv.org/abs/2012.03153 · HTML: https://mlsanity.com/paper/2012.03153/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2020.

## Built on by (most cited first, 6 of 6)

- [When Does Confidence-Based Cascade Deferral Suffice?](https://mlsanity.com/paper/2307.02764/) (2023)
- [Layer Pruning on Demand with Intermediate CTC](https://mlsanity.com/paper/2106.09216/) (2021)
- [Slicing Vision Transformer for Flexible Inference](https://mlsanity.com/paper/2412.04786/) (2024)
- [Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search](https://mlsanity.com/paper/2210.01384/) (2022)
- [SteppingNet: A Stepping Neural Network with Incremental Accuracy Enhancement](https://mlsanity.com/paper/2211.14926/) (2022)
- [Your Transformer Is Secretly Elastic](https://mlsanity.com/paper/2610.20588/) (2026)

## Abstract (arXiv)

Despite remarkable improvements in speed and accuracy, convolutional neural networks (CNNs) still typically operate as monolithic entities at inference time. This poses a challenge for resource-constrained practical applications, where both computational budgets and performance needs can vary with the situation. To address these constraints, we propose the Any-Width Network (AWN), an adjustable-width CNN architecture and associated training routine that allow for fine-grained control over speed and accuracy during inference. Our key innovation is the use of lower-triangular weight matrices which explicitly address width-varying batch statistics while being naturally suited for multi-width operations. We also show that this design facilitates an efficient training routine based on random width sampling. We empirically demonstrate that our proposed AWNs compare favorably to existing methods while providing maximally granular control during inference.