# Learning Architectures for Binary Networks

arXiv: https://arxiv.org/abs/2002.06963 · HTML: https://mlsanity.com/paper/2002.06963/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Feb 2020.

## Built on by (most cited first, 13 of 13)

- [Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network](https://mlsanity.com/paper/2103.09377/) (2021)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation](https://mlsanity.com/paper/2108.04541/) (2021)
- [A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network](https://mlsanity.com/paper/2403.03739/) (2024)
- [BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements](https://mlsanity.com/paper/2110.08562/) (2021)
- [The Binary Quantized Neural Network for Dense Prediction via Specially Designed Upsampling and Attention](https://mlsanity.com/paper/2405.17776/) (2024)
- [Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer](https://mlsanity.com/paper/2104.00210/) (2021)
- [DASS: Differentiable Architecture Search for Sparse neural networks](https://mlsanity.com/paper/2207.06968/) (2022)
- [Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning](https://mlsanity.com/paper/2110.08851/) (2021)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs](https://mlsanity.com/paper/2306.15390/) (2023)
- [Long-Tailed Recognition on Binary Networks by Calibrating A Pre-trained Model](https://mlsanity.com/paper/2404.00285/) (2024)
- [NAS-BNN: Neural Architecture Search for Binary Neural Networks](https://mlsanity.com/paper/2408.15484/) (2024)

## Abstract (arXiv)

Backbone architectures of most binary networks are well-known floating point architectures such as the ResNet family. Questioning that the architectures designed for floating point networks would not be the best for binary networks, we propose to search architectures for binary networks (BNAS) by defining a new search space for binary architectures and a novel search objective. Specifically, based on the cell based search method, we define the new search space of binary layer types, design a new cell template, and rediscover the utility of and propose to use the Zeroise layer instead of using it as a placeholder. The novel search objective diversifies early search to learn better performing binary architectures. We show that our proposed method searches architectures with stable training curves despite the quantization error inherent in binary networks. Quantitative analyses demonstrate that our searched architectures outperform the architectures used in state-of-the-art binary networks and outperform or perform on par with state-of-the-art binary networks that employ various techniques other than architectural changes.