HyBNN and FedHyBNN: (Federated) Hybrid Binary Neural Networks
Not on the map. Why?
Abstract (arXiv)
Binary Neural Networks (BNNs), neural networks with weights and activations constrained to -1(0) and +1, are an alternative to deep neural networks which offer faster training, lower memory consumption and lightweight models, ideal for use in resource constrained devices while being able to utilize the architecture of their deep neural network counterpart. However, the input binarization step used in BNNs causes a severe accuracy loss. In this paper, we introduce a novel hybrid neural network architecture, Hybrid Binary Neural Network (HyBNN), consisting of a task-independent, general, full-precision variational autoencoder with a binary latent space and a task specific binary neural network that is able to greatly limit the accuracy loss due to input binarization by using the full precision variational autoencoder as a feature extractor. We use it to combine the state-of-the-art accuracy of deep neural networks with the much faster training time, quicker test-time inference and power efficiency of binary neural networks. We show that our proposed system is able to very significantly outperform a vanilla binary neural network with input binarization. We also introduce FedHyBNN, a highly communication efficient federated counterpart to HyBNN and demonstrate that it is able to reach the same accuracy as its non-federated equivalent. We make our source code, experimental parameters and models available at: https://anonymous.4open.science/r/HyBNN.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 19 May 2022.
Builds on
21 of 29 references.
References · 21
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
- 2018A Survey on Deep Transfer Learningcited by 419
- 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 195
- 2020Binary Neural Networks: A Surveycited by 190
- 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
- 2019Quantization Networkscited by 153
- 2017Towards Accurate Binary Convolutional Neural Networkcited by 149
9 more
- 2019XNOR-Net++: Improved Binary Neural Networkscited by 97
- 2017WRPN: Wide Reduced-Precision Networkscited by 51
- 2018Self-Supervised Video Hashing with Hierarchical Binary Auto-encodercited by 34
- 2017Performance Guaranteed Network Acceleration via High-Order Residual Quantizationcited by 26
- 2018From Hashing to CNNs: Training BinaryWeight Networks via Hashingcited by 23
- 2019Accurate and Compact Convolutional Neural Networks with Trained Binarizationcited by 20
- 2015Hashing with binary autoencoderscited by 9
- 2020BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learningcited by 9
- 2017Transfer Learning with Binary Neural Networkscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021A comprehensive review of Binary Neural Network
- 2024CycleBNN: Cyclic Precision Training in Binary Neural Networks
- 2023Binary domain generalization for sparsifying binary neural networks
- 2021Dynamic Binary Neural Network by learning channel-wise thresholds
- 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
- 2024NAS-BNN: Neural Architecture Search for Binary Neural Networks
- 2021BCNN: Binary Complex Neural Network
- 2024Training Multi-Layer Binary Neural Networks With Local Binary Error Signals