mlsanityData to 14 Oct 2026

CycleBNN: Cyclic Precision Training in Binary Neural Networks

arXiv 2410.00050 ↗First posted 28 Sep 2024cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper works on Binary Neural Networks (BNNs), a promising avenue for efficient deep learning, offering significant reductions in computational overhead and memory footprint to full precision networks. However, the challenge of energy-intensive training and the drop in performance have been persistent issues. Tackling the challenge, prior works focus primarily on task-related inference optimization. Unlike prior works, this study offers an innovative methodology integrating BNNs with cyclic precision training, introducing the CycleBNN. This approach is designed to enhance training efficiency while minimizing the loss in performance. By dynamically adjusting precision in cycles, we achieve a convenient trade-off between training efficiency and model performance. This emphasizes the potential of our method in energy-constrained training scenarios, where data is collected onboard and paves the way for sustainable and efficient deep learning architectures. To gather insights on CycleBNN's efficiency, we conduct experiments on ImageNet, CIFAR-10, and PASCAL-VOC, obtaining competitive performances while using 96.09% less operations during training on ImageNet, 88.88% on CIFAR-10 and 96.09% on PASCAL-VOC. Finally, CycleBNN offers a path towards faster, more accessible training of efficient networks, accelerating the development of practical applications. The PyTorch code is available at urlhttps://github.com/fedeloper/CycleBNN/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

44 of 61 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  10. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  11. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  12. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
32 more
  1. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  2. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 235
  3. 2016Ternary Weight Networkscited by 204
  4. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  5. 2017Towards Accurate Binary Convolutional Neural Networkcited by 149
  6. 2020Rotated Binary Neural Networkcited by 77
  7. 2019Regularizing Activation Distribution for Training Binarized Deep Networkscited by 61
  8. 2016Loss-aware Binarization of Deep Networkscited by 56
  9. 2021ReCU: Reviving the Dead Weights in Binary Neural Networkscited by 52
  10. 2020Searching for Low-Bit Weights in Quantized Neural Networkscited by 44
  11. 2018Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?cited by 39
  12. 2020Training Binary Neural Networks through Learning with Noisy Supervisioncited by 39
  13. 2021Learning Frequency Domain Approximation for Binary Neural Networkscited by 38
  14. 2022AdaBin: Improving Binary Neural Networks with Adaptive Binary Setscited by 38
  15. 2020BiDet: An Efficient Binarized Object Detectorcited by 33
  16. 2020BiPointNet: Binary Neural Network for Point Cloudscited by 30
  17. 2024Faster Than Lies: Real-time Deepfake Detection using Binary Neural Networkscited by 27
  18. 2019Back to Simplicity: How to Train Accurate BNNs from Scratch?cited by 24
  19. 2020Fractional Skipping: Towards Finer-Grained Dynamic CNN Inferencecited by 23
  20. 2020FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Trainingcited by 23
  21. 2021CPT: Efficient Deep Neural Network Training via Cyclic Precisioncited by 19
  22. 2021SiMaN: Sign-to-Magnitude Network Binarizationcited by 18
  23. 2023BinaryViT: Pushing Binary Vision Transformers Towards Convolutional Modelscited by 17
  24. 2020Improving Accuracy of Binary Neural Networks using Unbalanced Activation Distributioncited by 15
  25. 2022Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticketcited by 14
  26. 2021Training Dynamical Binary Neural Networks with Equilibrium Propagationcited by 13
  27. 2022IDa-Det: An Information Discrepancy-aware Distillation for 1-bit Detectorscited by 12
  28. 2021Enabling Binary Neural Network Training on the Edgecited by 11
  29. 2023A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edgecited by 11
  30. 2022BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performancecited by 4
  31. 2023SCALES: Boost Binary Neural Network for Image Super-Resolution with Efficient Scalingscited by 4
  32. 2022LDP: Learnable Dynamic Precision for Efficient Deep Neural Network Training and Inferencecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Better Schedules for Low Precision Training of Deep Neural Networks
  2. 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
  3. 2021A comprehensive review of Binary Neural Network
  4. 2021BoolNet: Minimizing The Energy Consumption of Binary Neural Networks
  5. 2023Optimizing data-flow in Binary Neural Networks
  6. 2022HyBNN and FedHyBNN: (Federated) Hybrid Binary Neural Networks
  7. 2023On-Device Learning with Binary Neural Networks
  8. 2021How Do Adam and Training Strategies Help BNNs Optimization?