mlsanityData to 7 Oct 2026

APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Cores

arXiv 2106.12169 ↗First posted 23 Jun 2021cs.DC cs.AI cs.ARcited by 12

Not on the map. Why?

Abstract (arXiv)

Over the years, accelerating neural networks with quantization has been widely studied. Unfortunately, prior efforts with diverse precisions (e.g., 1-bit weights and 2-bit activations) are usually restricted by limited precision support on GPUs (e.g., int1 and int4). To break such restrictions, we introduce the first Arbitrary Precision Neural Network framework (APNN-TC) to fully exploit quantization benefits on Ampere GPU Tensor Cores. Specifically, APNN-TC first incorporates a novel emulation algorithm to support arbitrary short bit-width computation with int1 compute primitives and XOR/AND Boolean operations. Second, APNN-TC integrates arbitrary precision layer designs to efficiently map our emulation algorithm to Tensor Cores with novel batching strategies and specialized memory organization. Third, APNN-TC embodies a novel arbitrary precision NN design to minimize memory access across layers and further improve performance. Extensive evaluations show that APNN-TC can achieve significant speedup over CUTLASS kernels and various NN models, such as ResNet and VGG.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 31 of its 53 references that are in the map. First posted on arXiv 23 Jun 2021.

Builds on

31 of 53 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  12. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
19 more
  1. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  2. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  3. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  4. 2015Quantized Convolutional Neural Networks for Mobile Devicescited by 212
  5. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 115
  6. 2020Tracking by Instance Detection: A Meta-Learning Approachcited by 61
  7. 2019AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Ratescited by 49
  8. 2019PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devicescited by 48
  9. 2020Searching for Low-Bit Weights in Quantized Neural Networkscited by 43
  10. 2018Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarkingcited by 35
  11. 2019Training Quantized Neural Networks with a Full-precision Auxiliary Modulecited by 35
  12. 2020ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalizationcited by 20
  13. 2019Dissecting the NVidia Turing T4 GPU via Microbenchmarkingcited by 15
  14. 2020Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUscited by 12
  15. 2017Embedded Binarized Neural Networkscited by 8
  16. 2018Modeling Deep Learning Accelerator Enabled GPUscited by 8
  17. 2021BinaryCoP: Binary Neural Network-based COVID-19 Face-Mask Wear and Positioning Predictor on Edge Devicescited by 7
  18. 2020Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimizationcited by 2
  19. 2021DSXplore: Optimizing Convolutional Neural Networks via Sliding-Channel Convolutionscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 2; 2024: 2; 2025: 3.

12 papers.

  1. 2022NIPQ: Noise proxy-based Integrated Pseudo-Quantizationcited by 35
  2. 2022Efficient Quantized Sparse Matrix Operations on Tensor Corescited by 16
  3. 2022REx: Data-Free Residual Quantization Error Expansioncited by 6
  4. 2023BitGNN: Unleashing the Performance Potential of Binary Graph Neural Networks on GPUscited by 4
  5. 2024Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Corescited by 3
  6. 2025Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computationcited by 3
  7. 2024FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Corescited by 2
  8. 2025FlexQ: Efficient Post-training INT6 Quantization for LLM Serving via Algorithm-System Co-Designcited by 2
  9. 2022Learning from distinctive candidates to optimize reduced-precision convolution program on tensor corescited by 1
  10. 2025APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Accelerationcited by 1
  11. 2022Faith: An Efficient Framework for Transformer Verification on GPUscited by 0
  12. 2023PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansionscited by 0