# Regularized Binary Network Training

arXiv: https://arxiv.org/abs/1812.11800 · HTML: https://mlsanity.com/paper/1812.11800/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2018.

## Built on by (most cited first, 14 of 14)

- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy](https://mlsanity.com/paper/2103.13813/) (2021)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [On the Opportunities of Green Computing: A Survey](https://mlsanity.com/paper/2311.00447/) (2023)
- [Custom Gradient Estimators are Straight-Through Estimators in Disguise](https://mlsanity.com/paper/2405.05171/) (2024)
- [Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers](https://mlsanity.com/paper/2402.17710/) (2024)
- [A Bop and Beyond: A Second Order Optimizer for Binarized Neural Networks](https://mlsanity.com/paper/2104.05124/) (2021)
- [Learning Label Encodings for Deep Regression](https://mlsanity.com/paper/2303.02273/) (2023)
- [VC dimension of partially quantized neural networks in the overparametrized regime](https://mlsanity.com/paper/2110.02456/) (2021)
- [How to train accurate BNNs for embedded systems?](https://mlsanity.com/paper/2206.12322/) (2022)
- [Binary Neural Networks for Large Language Model: A Survey](https://mlsanity.com/paper/2502.19008/) (2025)
- [Quantization of Neural Network Equalizers in Optical Fiber Transmission Experiments](https://mlsanity.com/paper/2310.05950/) (2023)
- [A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs](https://mlsanity.com/paper/2412.19869/) (2024)
- [Cauchy-Schwarz Regularizers](https://mlsanity.com/paper/2503.01639/) (2025)

## Abstract (arXiv)

There is a significant performance gap between Binary Neural Networks (BNNs) and floating point Deep Neural Networks (DNNs). We propose to improve the binary training method, by introducing a new regularization function that encourages training weights around binary values. In addition, we add trainable scaling factors to our regularization functions. Additionally, an improved approximation of the derivative of the sign activation function in the backward computation. These modifications are based on linear operations that are easily implementable into the binary training framework. Experimental results on ImageNet shows our method outperforms the traditional BNN method and XNOR-net.