# End-to-end fully-binarized network design: from Generic Learned Thermometer to Block Pruning

arXiv: https://arxiv.org/abs/2505.13462 · HTML: https://mlsanity.com/paper/2505.13462/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 5 May 2025.

## Builds on (most cited first, 15 of 15)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [Visual Wake Words Dataset](https://mlsanity.com/paper/1906.05721/) (2019)
- [Differentiable Joint Pruning and Quantization for Hardware Efficiency](https://mlsanity.com/paper/2007.10463/) (2020)
- [FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations](https://mlsanity.com/paper/2012.12206/) (2020)
- [Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training](https://mlsanity.com/paper/2308.06689/) (2023)
- [AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networks](https://mlsanity.com/paper/2112.02880/) (2021)
- [Binary Input Layer: Training of CNN models with binary input data](https://mlsanity.com/paper/1812.03410/) (2018)
- [Depth Pruning with Auxiliary Networks for TinyML](https://mlsanity.com/paper/2204.10546/) (2022)
- [Histogram-Equalized Quantization for logic-gated Residual Neural Networks](https://mlsanity.com/paper/2501.04517/) (2025)
- [Input Layer Binarization with Bit-Plane Encoding](https://mlsanity.com/paper/2305.02885/) (2023)

## Abstract (arXiv)

Existing works on Binary Neural Network (BNN) mainly focus on model's weights and activations while discarding considerations on the input raw data. This article introduces Generic Learned Thermometer (GLT), an encoding technique to improve input data representation for BNN, relying on learning non linear quantization thresholds. This technique consists in multiple data binarizations which can advantageously replace a conventional Analog to Digital Conversion (ADC) that uses natural binary coding. Additionally, we jointly propose a compact topology with light-weight grouped convolutions being trained thanks to block pruning and Knowledge Distillation (KD), aiming at reducing furthermore the model size so as its computational complexity. We show that GLT brings versatility to the BNN by intrinsically performing global tone mapping, enabling significant accuracy gains in practice (demonstrated by simulations on the STL-10 and VWW datasets). Moreover, when combining GLT with our proposed block-pruning technique, we successfully achieve lightweight (under 1Mb), fully-binarized models with limited accuracy degradation while being suitable for in-sensor always-on inference use cases.