# Searching Similarity Measure for Binarized Neural Networks

arXiv: https://arxiv.org/abs/2206.03325 · HTML: https://mlsanity.com/paper/2206.03325/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 5 Jun 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [XNOR-Net++: Improved Binary Neural Networks](https://mlsanity.com/paper/1909.13863/) (2019)
- [Defensive Quantization: When Efficiency Meets Robustness](https://mlsanity.com/paper/1904.08444/) (2019)
- [LipNet: End-to-End Sentence-level Lipreading](https://mlsanity.com/paper/1611.01599/) (2016)
- [Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning](https://mlsanity.com/paper/1710.07654/) (2017)
- [Achieving Human Parity in Conversational Speech Recognition](https://mlsanity.com/paper/1610.05256/) (2016)
- [Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization](https://mlsanity.com/paper/1906.02107/) (2019)
- [Loss-aware Binarization of Deep Networks](https://mlsanity.com/paper/1611.01600/) (2016)
- [Evolving Normalization-Activation Layers](https://mlsanity.com/paper/2004.02967/) (2020)
- [Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?](https://mlsanity.com/paper/1806.07550/) (2018)
- [Efficient Super Resolution Using Binarized Neural Network](https://mlsanity.com/paper/1812.06378/) (2018)
- [Performance Guaranteed Network Acceleration via High-Order Residual Quantization](https://mlsanity.com/paper/1708.08687/) (2017)
- [From Hashing to CNNs: Training BinaryWeight Networks via Hashing](https://mlsanity.com/paper/1802.02733/) (2018)
- [Self-Binarizing Networks](https://mlsanity.com/paper/1902.00730/) (2019)
- [Deep Local Video Feature for Action Recognition](https://mlsanity.com/paper/1701.07368/) (2017)
- [Modeling and Propagating CNNs in a Tree Structure for Visual Tracking](https://mlsanity.com/paper/1608.07242/) (2016)
- [BinaryCoP: Binary Neural Network-based COVID-19 Face-Mask Wear and Positioning Predictor on Edge Devices](https://mlsanity.com/paper/2102.03456/) (2021)

## Abstract (arXiv)

Being a promising model to be deployed in resource-limited devices, Binarized Neural Networks (BNNs) have drawn extensive attention from both academic and industry. However, comparing to the full-precision deep neural networks (DNNs), BNNs suffer from non-trivial accuracy degradation, limiting its applicability in various domains. This is partially because existing network components, such as the similarity measure, are specially designed for DNNs, and might be sub-optimal for BNNs. In this work, we focus on the key component of BNNs -- the similarity measure, which quantifies the distance between input feature maps and filters, and propose an automatic searching method, based on genetic algorithm, for BNN-tailored similarity measure. Evaluation results on Cifar10 and Cifar100 using ResNet, NIN and VGG show that most of the identified similarty measure can achieve considerable accuracy improvement (up to 3.39%) over the commonly-used cross-correlation approach.