mlsanityData to 14 Oct 2026

BinaryConnect: Training Deep Neural Networks with binary weights during propagations

arXiv 1511.00363 ↗First posted 2 Nov 2015cs.LG cs.CV cs.NEcited by 592

Not on the map. Why?

Abstract (arXiv)

Deep Neural Networks (DNN) have achieved state-of-the-art results in a wide range of tasks, with the best results obtained with large training sets and large models. In the past, GPUs enabled these breakthroughs because of their greater computational speed. In the future, faster computation at both training and test time is likely to be crucial for further progress and for consumer applications on low-power devices. As a result, there is much interest in research and development of dedicated hardware for Deep Learning (DL). Binary weights, i.e., weights which are constrained to only two possible values (e.g. -1 or 1), would bring great benefits to specialized DL hardware by replacing many multiply-accumulate operations by simple accumulations, as multipliers are the most space and power-hungry components of the digital implementation of neural networks. We introduce BinaryConnect, a method which consists in training a DNN with binary weights during the forward and backward propagations, while retaining precision of the stored weights in which gradients are accumulated. Like other dropout schemes, we show that BinaryConnect acts as regularizer and we obtain near state-of-the-art results with BinaryConnect on the permutation-invariant MNIST, CIFAR-10 and SVHN.

Key facts. Cited by 592 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 160; 2022: 117; 2023: 125; 2024: 75; 2025: 69; 2026: 46.

592 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  3. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  4. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
  5. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  6. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  7. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  8. 2021I-BERT: Integer-only BERT Quantizationcited by 255
  9. 2022FP8 Formats for Deep Learningcited by 196
  10. 2023EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingScited by 196
  11. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 175
  12. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
38 more
  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  2. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  3. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 129
  4. 2021The Bayesian Learning Rulecited by 99
  5. 2021A Survey of Modern Deep Learning based Object Detection Modelscited by 98
  6. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  7. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  8. 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
  9. 2023Stable and low-precision training for large-scale vision-language modelscited by 75
  10. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  11. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 64
  12. 2021hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devicescited by 62
  13. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  14. 2021A Comprehensive Survey on Hardware-Aware Neural Architecture Searchcited by 61
  15. 2021ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Trainingcited by 61
  16. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  17. 2023SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficientcited by 59
  18. 2022Masked Wavelet Representation for Compact Neural Radiance Fieldscited by 57
  19. 2021Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Networkcited by 55
  20. 2023Memory Efficient Optimizers with 4-bit Statescited by 54
  21. 2021ReCU: Reviving the Dead Weights in Binary Neural Networkscited by 52
  22. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  23. 2021A Survey on Green Deep Learningcited by 52
  24. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 45
  25. 2021Categorical Foundations of Gradient-Based Learningcited by 44
  26. 2022Blueprint Separable Residual Network for Efficient Image Super-Resolutioncited by 44
  27. 2021Fast convolutional neural networks on FPGAs with hls4mlcited by 42
  28. 2021Scalable Rule-Based Representation Learning for Interpretable Classificationcited by 40
  29. 2022NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Resultscited by 40
  30. 2023Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Modelscited by 40
  31. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 39
  32. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 39
  33. 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
  34. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  35. 2022Rare Gems: Finding Lottery Tickets at Initializationcited by 34
  36. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 33
  37. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  38. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32

50 most cited of 592 All 592 (JSON)