# BATS: Binary ArchitecTure Search

arXiv: https://arxiv.org/abs/2003.01711 · HTML: https://mlsanity.com/paper/2003.01711/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 26 of 26)

- [Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network](https://mlsanity.com/paper/2103.09377/) (2021)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)
- [Lightweight Pixel Difference Networks for Efficient Visual Representation Learning](https://mlsanity.com/paper/2402.00422/) (2024)
- [Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks](https://mlsanity.com/paper/2104.07085/) (2021)
- [Lipschitz Continuity Retained Binary Neural Network](https://mlsanity.com/paper/2207.06540/) (2022)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networks](https://mlsanity.com/paper/2201.02711/) (2022)
- [Quantum HyperNetworks: Training Binary Neural Networks in Quantum Superposition](https://mlsanity.com/paper/2301.08292/) (2023)
- [BoolNet: Minimizing The Energy Consumption of Binary Neural Networks](https://mlsanity.com/paper/2106.06991/) (2021)
- [QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise](https://mlsanity.com/paper/2208.14839/) (2022)
- [RBNN: Memory-Efficient Reconfigurable Deep Binary Neural Network with IP Protection for Internet of Things](https://mlsanity.com/paper/2105.03822/) (2021)
- [Improving Binary Neural Networks through Fully Utilizing Latent Weights](https://mlsanity.com/paper/2110.05850/) (2021)
- [BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements](https://mlsanity.com/paper/2110.08562/) (2021)
- [DASS: Differentiable Architecture Search for Sparse neural networks](https://mlsanity.com/paper/2207.06968/) (2022)
- [Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning](https://mlsanity.com/paper/2110.08851/) (2021)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis](https://mlsanity.com/paper/2303.15493/) (2023)
- [ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less Neural Networks](https://mlsanity.com/paper/2204.05113/) (2022)
- [LAB: Learnable Activation Binarizer for Binary Neural Networks](https://mlsanity.com/paper/2210.13858/) (2022)
- [DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs](https://mlsanity.com/paper/2306.15390/) (2023)
- [SupeRBNN: Randomized Binary Neural Network Using Adiabatic Superconductor Josephson Devices](https://mlsanity.com/paper/2309.12212/) (2023)
- [Hamming Encoder: Mining Discriminative k-mers for Discrete Sequence Classification](https://mlsanity.com/paper/2310.10321/) (2023)
- [Long-Tailed Recognition on Binary Networks by Calibrating A Pre-trained Model](https://mlsanity.com/paper/2404.00285/) (2024)
- [NAS-BNN: Neural Architecture Search for Binary Neural Networks](https://mlsanity.com/paper/2408.15484/) (2024)

## Abstract (arXiv)

This paper proposes Binary ArchitecTure Search (BATS), a framework that drastically reduces the accuracy gap between binary neural networks and their real-valued counterparts by means of Neural Architecture Search (NAS). We show that directly applying NAS to the binary domain provides very poor results. To alleviate this, we describe, to our knowledge, for the first time, the 3 key ingredients for successfully applying NAS to the binary domain. Specifically, we (1) introduce and design a novel binary-oriented search space, (2) propose a new mechanism for controlling and stabilising the resulting searched topologies, (3) propose and validate a series of new search strategies for binary networks that lead to faster convergence and lower search times. Experimental results demonstrate the effectiveness of the proposed approach and the necessity of searching in the binary space directly. Moreover, (4) we set a new state-of-the-art for binary neural networks on CIFAR10, CIFAR100 and ImageNet datasets. Code will be made available https://github.com/1adrianb/binary-nas