mlsanityData to 7 Oct 2026

Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization

arXiv 1906.02107 ↗First posted 5 Jun 2019cs.LG stat.MLcited by 58

Not on the map. Why?

Abstract (arXiv)

Optimization of Binarized Neural Networks (BNNs) currently relies on real-valued latent weights to accumulate small update steps. In this paper, we argue that these latent weights cannot be treated analogously to weights in real-valued networks. Instead their main role is to provide inertia during training. We interpret current methods in terms of inertia and provide novel insights into the optimization of BNNs. We subsequently introduce the first optimizer specifically designed for BNNs, Binary Optimizer (Bop), and demonstrate its performance on CIFAR-10 and ImageNet. Together, the redefinition of latent weights as inertia and the introduction of Bop enable a better understanding of BNN optimization and open up the way for further improvements in training methodologies for BNNs. Code is available at: https://github.com/plumerai/rethinking-bnn-optimization

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 12; 2023: 11; 2024: 9; 2025: 3; 2026: 6.

58 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 137
  3. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  4. 2021The Bayesian Learning Rulecited by 95
  5. 2023PB-LLM: Partially Binarized Large Language Modelscited by 80
  6. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  7. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  8. 2021ReCU: Reviving the Dead Weights in Binary Neural Networkscited by 51
  9. 2021How Do Adam and Training Strategies Help BNNs Optimization?cited by 46
  10. 2021A comprehensive review of Binary Neural Networkcited by 44
  11. 2021Learning Frequency Domain Approximation for Binary Neural Networkscited by 38
  12. 2023BiBench: Benchmarking and Analyzing Network Binarizationcited by 22
38 more
  1. 2023Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Trainingcited by 19
  2. 2021SiMaN: Sign-to-Magnitude Network Binarizationcited by 18
  3. 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalizationcited by 17
  4. 2022Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticketcited by 13
  5. 2021Enabling Binary Neural Network Training on the Edgecited by 11
  6. 2021Demystifying and Generalizing BinaryConnectcited by 11
  7. 2021AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networkscited by 11
  8. 2022TerViT: An Efficient Ternary Vision Transformercited by 10
  9. 2021Equal Bits: Enforcing Equally Distributed Binary Network Weightscited by 9
  10. 2023Globally Optimal Training of Neural Networks with Threshold Activation Functionscited by 9
  11. 2021BoolNet: Minimizing The Energy Consumption of Binary Neural Networkscited by 6
  12. 2022HideNseek: Federated Lottery Ticket via Server-side Pruning and Sign Supermaskcited by 6
  13. 2024Custom Gradient Estimators are Straight-Through Estimators in Disguisecited by 6
  14. 2025Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanismcited by 5
  15. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Modelscited by 5
  16. 2023Neural-based classification rule learning for sequential datacited by 4
  17. 2024Enabling On-device Continual Learning with Binary Neural Networkscited by 4
  18. 2024Understanding Neural Network Binarization with Forward and Backward Proximal Quantizerscited by 4
  19. 2021A Bop and Beyond: A Second Order Optimizer for Binarized Neural Networkscited by 3
  20. 2021Improving Binary Neural Networks through Fully Utilizing Latent Weightscited by 3
  21. 2022Hierarchical Symbolic Reasoning in Hyperbolic Space for Deep Discriminative Modelscited by 3
  22. 2024BOLD: Boolean Logic Deep Learningcited by 3
  23. 2021BCNN: Binary Complex Neural Networkcited by 2
  24. 2022CTMQ: Cyclic Training of Convolutional Neural Networks with Multiple Quantization Stepscited by 2
  25. 2022E2FIF: Push the limit of Binarized Deep Imagery Super-resolution using End-to-end Full-precision Information Flowcited by 2
  26. 2023Straight-Through meets Sparse Recovery: the Support Exploration Algorithmcited by 2
  27. 2023Optimizing data-flow in Binary Neural Networkscited by 2
  28. 2023Boolean Variation and Boolean Logic BackPropagationcited by 2
  29. 2024OvSW: Overcoming Silent Weights for Accurate Binary Neural Networkscited by 2
  30. 2022Confidence Dimension for Deep Learning based on Hoeffding Inequality and Relative Evaluationcited by 1
  31. 2022GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithmcited by 1
  32. 2022Binary Early-Exit Network for Adaptive Inference on Low-Resource Devicescited by 1
  33. 2022How to train accurate BNNs for embedded systems?cited by 1
  34. 2024Scheduling Weight Transitions for Quantization-Aware Trainingcited by 1
  35. 2024Training Multi-Layer Binary Neural Networks With Local Binary Error Signalscited by 1
  36. 2021SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Trainingcited by 0
  37. 2022Searching Similarity Measure for Binarized Neural Networkscited by 0
  38. 2023Understanding weight-magnitude hyperparameters in training binary networkscited by 0

50 most cited of 58 All 58 (JSON)