mlsanityData to 14 Oct 2026

Demystifying and Generalizing BinaryConnect

arXiv 2110.13220 ↗First posted 25 Oct 2021cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

BinaryConnect (BC) and its many variations have become the de facto standard for neural network quantization. However, our understanding of the inner workings of BC is still quite limited. We attempt to close this gap in four different aspects: (a) we show that existing quantization algorithms, including post-training quantization, are surprisingly similar to each other; (b) we argue for proximal maps as a natural family of quantizers that is both easy to design and analyze; (c) we refine the observation that BC is a special case of dual averaging, which itself is a special case of the generalized conditional gradient algorithm; (d) consequently, we propose ProxConnect (PC) as a generalization of BC and we prove its convergence properties by exploiting the established connections. We conduct experiments on CIFAR-10 and ImageNet, and verify that PC achieves competitive performance.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 35 of its 64 references that are in the map. First posted on arXiv 25 Oct 2021.

Builds on

35 of 64 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  4. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  5. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  6. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  7. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  8. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  9. 2015Deep Learning with Limited Numerical Precisioncited by 376
  10. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 295
  11. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  12. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
23 more
  1. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  2. 2016Trained Ternary Quantizationcited by 208
  3. 2016Ternary Weight Networkscited by 204
  4. 2020Binary Neural Networks: A Surveycited by 190
  5. 2020Characterising Bias in Compressed Modelscited by 167
  6. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 112
  7. 2017Training Quantized Nets: A Deeper Understandingcited by 75
  8. 2018A Survey on Methods and Theories of Quantized Neural Networkscited by 75
  9. 2017Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMMcited by 61
  10. 2019Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimizationcited by 60
  11. 2018ProxQuant: Quantized Neural Networks via Proximal Operatorscited by 59
  12. 2020Training Binary Neural Networks through Learning with Noisy Supervisioncited by 39
  13. 2018Alternating Multi-bit Quantization for Recurrent Neural Networkscited by 31
  14. 2017Learning Discrete Weights Using the Local Reparameterization Trickcited by 29
  15. 2018BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weightscited by 24
  16. 2017The High-Dimensional Geometry of Binary Neural Networkscited by 22
  17. 2018Probabilistic Binary Neural Networkscited by 21
  18. 2019Mirror Descent View for Neural Network Quantizationcited by 16
  19. 2018On the Universal Approximability and Complexity Bounds of Quantized ReLU Neural Networkscited by 12
  20. 2018Deep Frank-Wolfe For Neural Network Optimizationcited by 11
  21. 2016Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detectioncited by 8
  22. 2020Self-Concordant Analysis of Frank-Wolfe Algorithmscited by 8
  23. 2018On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 1; 2024: 1; 2025: 5; 2026: 2.

11 papers.

  1. 2022Spartan: Differentiable Sparsity via Regularized Transportationcited by 8
  2. 2023Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Datacited by 5
  3. 2025PARQ: Piecewise-Affine Regularized Quantizationcited by 5
  4. 2024Understanding Neural Network Binarization with Forward and Backward Proximal Quantizerscited by 4
  5. 2025High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimatorcited by 4
  6. 2025Beyond Discreteness: Sample Complexity Analysis of Straight-Through Estimator for 1-bit Quantizationcited by 2
  7. 2025Binary Neural Networks for Large Language Model: A Surveycited by 1
  8. 2026Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basincited by 1
  9. 2022Channel Pruning In Quantization-aware Training: An Adaptive Projection-gradient Descent-shrinkage-splitting Methodcited by 0
  10. 2025Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guaranteescited by 0
  11. 2026Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024QGen: On the Ability to Generalize in Quantization Aware Training
  2. 2022Post-training Quantization for Neural Networks with Provable Guarantees
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inference
  4. 2022AMED: Automatic Mixed-Precision Quantization for Edge Devices
  5. 2021Pareto-Optimal Quantized ResNet Is Mostly 4-bit
  6. 2022QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
  7. 2023MixQuant: Mixed Precision Quantization with a Bit-width Optimization Search
  8. 2024Gradient-Free Training of Quantized Neural Networks