mlsanityData to 14 Oct 2026

High-Fidelity Differential-information Driven Binary Vision Transformer

arXiv 2507.02222 ↗First posted 3 Jul 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The binarization of vision transformers (ViTs) offers a promising approach to addressing the trade-off between high computational/storage demands and the constraints of edge-device deployment. However, existing binary ViT methods often suffer from severe performance degradation or rely heavily on full-precision modules. To address these issues, we propose DIDB-ViT, a novel binary ViT that is highly informative while maintaining the original ViT architecture and computational efficiency. Specifically, we design an informative attention module incorporating differential information to mitigate information loss caused by binarization and enhance high-frequency retention. To preserve the fidelity of the similarity calculations between binary Q and K tensors, we apply frequency decomposition using the discrete Haar wavelet and integrate similarities across different frequencies. Additionally, we introduce an improved RPReLU activation function to restructure the activation distribution, expanding the model's representational capacity. Experimental results demonstrate that our DIDB-ViT significantly outperforms state-of-the-art network quantization methods in multiple ViT architectures, achieving superior image classification and segmentation performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

37 of 51 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  6. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  7. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  8. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  9. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  10. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  11. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  12. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 235
25 more
  1. 2020Binary Neural Networks: A Surveycited by 190
  2. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  3. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  4. 2017Towards Accurate Binary Convolutional Neural Networkcited by 149
  5. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  6. 2017Towards Effective Low-bitwidth Convolutional Neural Networkscited by 78
  7. 2020Rotated Binary Neural Networkcited by 77
  8. 2022Improving Vision Transformers by Revisiting High-frequency Componentscited by 76
  9. 2022BiBERT: Accurate Fully Binarized BERTcited by 73
  10. 2022BiT: Robustly Binarized Multi-distilled Transformercited by 52
  11. 2021PokeBNN: A Binary Pursuit of Lightweight Accuracycited by 49
  12. 2021Learning Frequency Domain Approximation for Binary Neural Networkscited by 38
  13. 2022AdaBin: Improving Binary Neural Networks with Adaptive Binary Setscited by 38
  14. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  15. 2023An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attentioncited by 28
  16. 2023Binarized Spectral Compressive Imagingcited by 23
  17. 2023BiBench: Benchmarking and Analyzing Network Binarizationcited by 22
  18. 2021SiMaN: Sign-to-Magnitude Network Binarizationcited by 18
  19. 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalizationcited by 17
  20. 2023BinaryViT: Pushing Binary Vision Transformers Towards Convolutional Modelscited by 17
  21. 2023Resilient Binary Neural Networkcited by 13
  22. 2023Bi-ViT: Pushing the Limit of Vision Transformer Quantizationcited by 13
  23. 2022Recurrent Bilinear Optimization for Binary Neural Networkscited by 12
  24. 2023GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samplescited by 4
  25. 2024Pathfinder for Low-altitude Aircraft with Binary Neural Networkcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023BinaryViT: Towards Efficient and Accurate Binary Vision Transformers
  2. 2025BHViT: Binarized Hybrid Vision Transformer
  3. 2026BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers
  4. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformer
  5. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
  6. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  7. 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking
  8. 2023TSPTQ-ViT: Two-scaled post-training quantization for vision transformer