# Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs

arXiv: https://arxiv.org/abs/2006.16578 · HTML: https://mlsanity.com/paper/2006.16578/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2020.

## Built on by (most cited first, 14 of 14)

- [BiViT: Extremely Compressed Binary Vision Transformer](https://mlsanity.com/paper/2211.07091/) (2022)
- [QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Core](https://mlsanity.com/paper/2111.09547/) (2021)
- [TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs](https://mlsanity.com/paper/2112.02052/) (2021)
- [Efficient Quantized Sparse Matrix Operations on Tensor Cores](https://mlsanity.com/paper/2209.06979/) (2022)
- [APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Cores](https://mlsanity.com/paper/2106.12169/) (2021)
- [BitGNN: Unleashing the Performance Potential of Binary Graph Neural Networks on GPUs](https://mlsanity.com/paper/2305.02522/) (2023)
- [Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores](https://mlsanity.com/paper/2409.17870/) (2024)
- [BCNN: Binary Complex Neural Network](https://mlsanity.com/paper/2104.10044/) (2021)
- [GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithm](https://mlsanity.com/paper/2206.03291/) (2022)
- [APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration](https://mlsanity.com/paper/2508.19087/) (2025)
- [HEP-BNN: A Framework for Finding Low-Latency Execution Configurations of BNNs on Heterogeneous Multiprocessor Platforms](https://mlsanity.com/paper/2301.05126/) (2023)
- [Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores](https://mlsanity.com/paper/2601.11660/) (2026)
- [Learning 1-Bit LiDAR-based Localization with Auxiliary Objective](https://mlsanity.com/paper/2606.27729/) (2026)
- [Self-Indexing Attention for Compression-Compatible Sparse Long-Context LLM Inference](https://mlsanity.com/paper/2609.13205/) (2026)

## Abstract (arXiv)

Despite foreseeing tremendous speedups over conventional deep neural networks, the performance advantage of binarized neural networks (BNNs) has merely been showcased on general-purpose processors such as CPUs and GPUs. In fact, due to being unable to leverage bit-level-parallelism with a word-based architecture, GPUs have been criticized for extremely low utilization (1%) when executing BNNs. Consequently, the latest tensorcores in NVIDIA Turing GPUs start to experimentally support bit computation. In this work, we look into this brand new bit computation capability and characterize its unique features. We show that the stride of memory access can significantly affect performance delivery and a data-format co-design is highly desired to support the tensorcores for achieving superior performance than existing software solutions without tensorcores. We realize the tensorcore-accelerated BNN design, particularly the major functions for fully-connect and convolution layers -- bit matrix multiplication and bit convolution. Evaluations on two NVIDIA Turing GPUs show that, with ResNet-18, our BTC-BNN design can process ImageNet at a rate of 5.6K images per second, 77% faster than state-of-the-art. Our BNN approach is released on https://github.com/pnnl/TCBNN.