mlsanityData to 14 Oct 2026

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1

arXiv 1602.02830 ↗First posted 9 Feb 2016cs.LGcited by 105

Not on the map. Why?

Abstract (arXiv)

We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time. At training-time the binary weights and activations are used for computing the parameters gradients. During the forward pass, BNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations, which is expected to substantially improve power-efficiency. To validate the effectiveness of BNNs we conduct two sets of experiments on the Torch7 and Theano frameworks. On both, BNNs achieved nearly state-of-the-art results over the MNIST, CIFAR-10 and SVHN datasets. Last but not least, we wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for training and running our BNNs is available on-line.

Key facts. Cited by 105 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 61; 2022: 30; 2023: 10; 2024: 2; 2025: 0; 2026: 2.

105 papers.

Sort:

By area:

  1. 2021Post-Training Quantization for Vision Transformercited by 278
  2. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  3. 2021I-BERT: Integer-only BERT Quantizationcited by 255
  4. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  5. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 129
  6. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  7. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  8. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  9. 2021AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networkscited by 59
  10. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 45
  11. 2022AdaBin: Improving Binary Neural Networks with Adaptive Binary Setscited by 38
  12. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
38 more
  1. 2021Content-Aware GAN Compressioncited by 28
  2. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 28
  3. 2021Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learningcited by 27
  4. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
  5. 2021Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networkscited by 25
  6. 2021Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolutioncited by 20
  7. 2021CPT: Efficient Deep Neural Network Training via Cyclic Precisioncited by 19
  8. 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalizationcited by 17
  9. 2022Lite Pose: Efficient Architecture Design for 2D Human Pose Estimationcited by 16
  10. 2021S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibrationcited by 15
  11. 2023Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edgecited by 15
  12. 2021Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descentcited by 14
  13. 2021Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networkscited by 14
  14. 2021Dynamic Binary Neural Network by learning channel-wise thresholdscited by 13
  15. 2022Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networkscited by 13
  16. 2022Weightless Neural Networks for Efficient Edge Inferencecited by 13
  17. 2022IDa-Det: An Information Discrepancy-aware Distillation for 1-bit Detectorscited by 12
  18. 2021Leaky Nets: Recovering Embedded Neural Network Models and Inputs through Simple Power and Timing Side-Channels -- Attacks and Defensescited by 11
  19. 2021FLASH: Fast Neural Architecture Search with Hardware Optimizationcited by 11
  20. 2021An In-Memory Analog Computing Co-Processor for Energy-Efficient CNN Inference on Mobile Devicescited by 9
  21. 2021FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applicationscited by 9
  22. 2021BinarizedAttack: Structural Poisoning Attacks to Graph-based Anomaly Detectioncited by 9
  23. 2021AHAR: Adaptive CNN for Energy-efficient Human Activity Recognition in Low-power Edge Devicescited by 8
  24. 2021Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNscited by 8
  25. 2022The fine line between dead neurons and sparsity in binarized spiking neural networkscited by 8
  26. 2021Recent Advances in Large Margin Learningcited by 7
  27. 2021Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compressioncited by 7
  28. 2021Communication-oriented Model Fine-tuning for Packet-loss Resilient Distributed Inference under Highly Lossy IoT Networkscited by 7
  29. 2021Constrained Differentially Private Federated Learning for Low-bandwidth Devicescited by 6
  30. 2022HideNseek: Federated Lottery Ticket via Server-side Pruning and Sign Supermaskcited by 6
  31. 2021Boosting Binary Masks for Multi-Domain Learning through Affine Transformationscited by 5
  32. 2021A 1D-CNN Based Deep Learning Technique for Sleep Apnea Detection in IoT Sensorscited by 5
  33. 2022Sub-mW Keyword Spotting on an MCU: Analog Binary Feature Extraction and Binary Neural Networkscited by 5
  34. 2022BiFSMN: Binary Neural Network for Keyword Spottingcited by 5
  35. 2022Binary Single-dimensional Convolutional Neural Network for Seizure Predictioncited by 5
  36. 2021End-to-End Learned Image Compression with Quantized Weights and Activationscited by 4
  37. 2022PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++cited by 4
  38. 2022ThreshNet: An Efficient DenseNet Using Threshold Mechanism to Reduce Connectionscited by 4

50 most cited of 105 All 105 (JSON)