# Improved training of binary networks for human pose estimation and image recognition

arXiv: https://arxiv.org/abs/1904.05868 · HTML: https://mlsanity.com/paper/1904.05868/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2019.

## Built on by (most cited first, 22 of 22)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [PokeBNN: A Binary Pursuit of Lightweight Accuracy](https://mlsanity.com/paper/2112.00133/) (2021)
- [How Do Adam and Training Strategies Help BNNs Optimization?](https://mlsanity.com/paper/2106.11309/) (2021)
- [Distribution-sensitive Information Retention for Accurate Binary Neural Network](https://mlsanity.com/paper/2109.12338/) (2021)
- [BiPer: Binary Neural Networks using a Periodic Function](https://mlsanity.com/paper/2404.01278/) (2024)
- [Self-Distribution Binary Neural Networks](https://mlsanity.com/paper/2103.02394/) (2021)
- [Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators](https://mlsanity.com/paper/2110.03549/) (2021)
- [Bimodal Distributed Binarized Neural Networks](https://mlsanity.com/paper/2204.02004/) (2022)
- [Improving Binary Neural Networks through Fully Utilizing Latent Weights](https://mlsanity.com/paper/2110.05850/) (2021)
- [Binarizing by Classification: Is soft function really necessary?](https://mlsanity.com/paper/2205.07433/) (2022)
- [Exact Backpropagation in Binary Weighted Networks with Group Weight Transformations](https://mlsanity.com/paper/2107.01400/) (2021)
- [How to train accurate BNNs for embedded systems?](https://mlsanity.com/paper/2206.12322/) (2022)
- [Signed Binary Weight Networks](https://mlsanity.com/paper/2211.13838/) (2022)
- [Bit-wise Training of Neural Network Weights](https://mlsanity.com/paper/2202.09571/) (2022)
- [IR2Net: Information Restriction and Information Recovery for Accurate Binary Neural Networks](https://mlsanity.com/paper/2210.02637/) (2022)
- [LAB: Learnable Activation Binarizer for Binary Neural Networks](https://mlsanity.com/paper/2210.13858/) (2022)
- [BiHRNet: A Binary high-resolution network for Human Pose Estimation](https://mlsanity.com/paper/2311.10296/) (2023)
- [PLUM: Improving Inference Efficiency By Leveraging Repetition-Sparsity Trade-Off](https://mlsanity.com/paper/2312.01581/) (2023)
- [What to Do When Your Discrete Optimization Is the Size of a Neural Network?](https://mlsanity.com/paper/2402.10339/) (2024)
- [NAS-BNN: Neural Architecture Search for Binary Neural Networks](https://mlsanity.com/paper/2408.15484/) (2024)
- [A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks](https://mlsanity.com/paper/2508.12609/) (2025)

## Abstract (arXiv)

Big neural networks trained on large datasets have advanced the state-of-the-art for a large variety of challenging problems, improving performance by a large margin. However, under low memory and limited computational power constraints, the accuracy on the same problems drops considerable. In this paper, we propose a series of techniques that significantly improve the accuracy of binarized neural networks (i.e networks where both the features and the weights are binary). We evaluate the proposed improvements on two diverse tasks: fine-grained recognition (human pose estimation) and large-scale image recognition (ImageNet classification). Specifically, we introduce a series of novel methodological changes including: (a) more appropriate activation functions, (b) reverse-order initialization, (c) progressive quantization, and (d) network stacking and show that these additions improve existing state-of-the-art network binarization techniques, significantly. Additionally, for the first time, we also investigate the extent to which network binarization and knowledge distillation can be combined. When tested on the challenging MPII dataset, our method shows a performance improvement of more than 4% in absolute terms. Finally, we further validate our findings by applying the proposed techniques for large-scale object recognition on the Imagenet dataset, on which we report a reduction of error rate by 4%.