mlsanityData to 14 Oct 2026

BiBench: Benchmarking and Analyzing Network Binarization

arXiv 2301.11233 ↗First posted 26 Jan 2023cs.CV cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Network binarization emerges as one of the most promising compression approaches offering extraordinary computation and memory savings by minimizing the bit-width. However, recent research has shown that applying existing binarization algorithms to diverse tasks, architectures, and hardware in realistic scenarios is still not straightforward. Common challenges of binarization, such as accuracy degradation and efficiency limitation, suggest that its attributes are not fully understood. To close this gap, we present BiBench, a rigorously designed benchmark with in-depth analysis for network binarization. We first carefully scrutinize the requirements of binarization in the actual production and define evaluation tracks and metrics for a comprehensive and fair investigation. Then, we evaluate and analyze a series of milestone binarization algorithms that function at the operator level and with extensive influence. Our benchmark reveals that 1) the binarized operator has a crucial impact on the performance and deployability of binarized networks; 2) the accuracy of binarization varies significantly across different learning tasks and neural architectures; 3) binarization has demonstrated promising efficiency potential on edge devices despite the limited hardware support. The results and analysis also lead to a promising paradigm for accurate and efficient binarization. We believe that BiBench will contribute to the broader adoption of binarization and serve as a foundation for future research. The code for our BiBench is released https://github.com/htqin/BiBench .

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 95 of its 144 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

95 of 144 references.

References · 95

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  10. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  11. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  12. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
38 more
  1. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  5. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  6. 2016Adversarial examples in the physical worldcited by 1,769
  7. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  10. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  11. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  12. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  13. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  14. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  15. 2019Learned Step Size Quantizationcited by 628
  16. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  17. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  18. 2015Deep Learning with Limited Numerical Precisioncited by 376
  19. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 306
  20. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 284
  21. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  22. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 235
  23. 2022CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusioncited by 223
  24. 2015Quantized Convolutional Neural Networks for Mobile Devicescited by 215
  25. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 203
  26. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 195
  27. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  28. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  29. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  30. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  31. 2017Towards Accurate Binary Convolutional Neural Networkcited by 149
  32. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  33. 2021Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical Worldcited by 127
  34. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 112
  35. 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
  36. 2019XNOR-Net++: Improved Binary Neural Networkscited by 97
  37. 2019Defensive Quantization: When Efficiency Meets Robustnesscited by 95
  38. 2020ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgettingcited by 85

50 most cited of 95 All 95 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 8; 2025: 3; 2026: 3.

22 papers.

Sort:

By area:

  1. 2023DDFM: Denoising Diffusion Model for Multi-Modality Image Fusioncited by 153
  2. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  3. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  4. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Modelscited by 21
  5. 2023RobustMQ: Benchmarking Robustness of Quantized Modelscited by 19
  6. 2024Binarized Diffusion Model for Image Super-Resolutioncited by 18
  7. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  8. 2023Cosine Similarity Knowledge Distillation for Individual Class Information Transfercited by 5
  9. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Modelscited by 5
  10. 2023GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samplescited by 4
  11. 2024BOLD: Boolean Logic Deep Learningcited by 3
  12. 2023Benchmarking the Robustness of Quantized Modelscited by 2
10 more
  1. 2024Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAscited by 1
  2. 2025BiVM: Accurate Binarized Neural Network for Efficient Video Mattingcited by 1
  3. 2026TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Flycited by 1
  4. 2023Neural Network Compression using Binarization and Few Full-Precision Weightscited by 0
  5. 2023ARBiBench: Benchmarking Adversarial Robustness of Binarized Neural Networkscited by 0
  6. 2024Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectorscited by 0
  7. 2024BEExformer: A Fast Inferencing Binarized Transformer with Early Exitscited by 0
  8. 2025High-Fidelity Differential-information Driven Binary Vision Transformercited by 0
  9. 2026BinaryDemoire: Moir'e-Aware Binarization for Image Demoir'eingcited by 0
  10. 2026FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Pruning Binarized Neural Networks: A Dedicated Framework and Globally Weighted Algorithms
  2. 2021Enabling Binary Neural Network Training on the Edge
  3. 2022Partial Binarization of Neural Networks for Budget-Aware Efficient Learning
  4. 2022LAB: Learnable Activation Binarizer for Binary Neural Networks
  5. 2021Equal Bits: Enforcing Equally Distributed Binary Network Weights
  6. 2023Input Layer Binarization with Bit-Plane Encoding
  7. 2024Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers
  8. 2023MST-compression: Compressing and Accelerating Binary Neural Networks with Minimum Spanning Tree