# High-Capacity Expert Binary Networks

arXiv: https://arxiv.org/abs/2010.03558 · HTML: https://mlsanity.com/paper/2010.03558/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 30 of 34)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network](https://mlsanity.com/paper/2103.09377/) (2021)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization](https://mlsanity.com/paper/2111.09136/) (2021)
- [F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization](https://mlsanity.com/paper/2202.05239/) (2022)
- [BiViT: Extremely Compressed Binary Vision Transformer](https://mlsanity.com/paper/2211.07091/) (2022)
- [Bit-Mixer: Mixed-precision networks with runtime bit-width selection](https://mlsanity.com/paper/2103.17267/) (2021)
- [BiBench: Benchmarking and Analyzing Network Binarization](https://mlsanity.com/paper/2301.11233/) (2023)
- [Lightweight Pixel Difference Networks for Efficient Visual Representation Learning](https://mlsanity.com/paper/2402.00422/) (2024)
- [0/1 Deep Neural Networks via Block Coordinate Descent](https://mlsanity.com/paper/2206.09379/) (2022)
- [INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold](https://mlsanity.com/paper/2204.07439/) (2022)
- [Mixed-Precision Neural Networks: A Survey](https://mlsanity.com/paper/2208.06064/) (2022)
- [Efficient Multitask Dense Predictor via Binarization](https://mlsanity.com/paper/2405.14136/) (2024)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [BoolNet: Minimizing The Energy Consumption of Binary Neural Networks](https://mlsanity.com/paper/2106.06991/) (2021)
- [Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators](https://mlsanity.com/paper/2110.03549/) (2021)
- [Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices](https://mlsanity.com/paper/2108.06128/) (2021)
- [Improving Binary Neural Networks through Fully Utilizing Latent Weights](https://mlsanity.com/paper/2110.05850/) (2021)
- [Compacting Binary Neural Networks by Sparse Kernel Selection](https://mlsanity.com/paper/2303.14470/) (2023)
- [Input Layer Binarization with Bit-Plane Encoding](https://mlsanity.com/paper/2305.02885/) (2023)
- [Binarizing by Classification: Is soft function really necessary?](https://mlsanity.com/paper/2205.07433/) (2022)
- [BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation](https://mlsanity.com/paper/2207.01394/) (2022)
- [RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating](https://mlsanity.com/paper/2207.09049/) (2022)
- [Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training](https://mlsanity.com/paper/2211.08544/) (2022)
- [Optimizing data-flow in Binary Neural Networks](https://mlsanity.com/paper/2304.00952/) (2023)
- [Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning](https://mlsanity.com/paper/2110.08851/) (2021)
- [How to train accurate BNNs for embedded systems?](https://mlsanity.com/paper/2206.12322/) (2022)
- [Bit-wise Training of Neural Network Weights](https://mlsanity.com/paper/2202.09571/) (2022)

## Abstract (arXiv)

Network binarization is a promising hardware-aware direction for creating efficient deep models. Despite its memory and computational advantages, reducing the accuracy gap between binary models and their real-valued counterparts remains an unsolved challenging research problem. To this end, we make the following 3 contributions: (a) To increase model capacity, we propose Expert Binary Convolution, which, for the first time, tailors conditional computing to binary networks by learning to select one data-specific expert binary filter at a time conditioned on input features. (b) To increase representation capacity, we propose to address the inherent information bottleneck in binary networks by introducing an efficient width expansion mechanism which keeps the binary operations within the same budget. (c) To improve network design, we propose a principled binary network growth mechanism that unveils a set of network topologies of favorable properties. Overall, our method improves upon prior work, with no increase in computational cost, by ~6 %, reaching a groundbreaking ~ 71% on ImageNet classification. Code will be made available hrefhttps://www.adrianbulat.com/binary-networkshere.