mlsanityData to 7 Oct 2026

DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables

arXiv 2304.09049 ↗First posted 18 Apr 2023cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

A lot of recent progress has been made in ultra low-bit quantization, promising significant improvements in latency, memory footprint and energy consumption on edge devices. Quantization methods such as Learned Step Size Quantization can achieve model accuracy that is comparable to full-precision floating-point baselines even with sub-byte quantization. However, it is extremely challenging to deploy these ultra low-bit quantized models on mainstream CPU devices because commodity SIMD (Single Instruction, Multiple Data) hardware typically supports no less than 8-bit precision. To overcome this limitation, we propose DeepGEMM, a lookup table based approach for the execution of ultra low-precision convolutional neural networks on SIMD hardware. The proposed method precomputes all possible products of weights and activations, stores them in a lookup table, and efficiently accesses them at inference time to avoid costly multiply-accumulate operations. Our 2-bit implementation outperforms corresponding 8-bit integer kernels in the QNNPACK framework by up to 1.74x on x86 platforms.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

14 of 24 references.

References · 14

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019Learned Step Size Quantizationcited by 604
  3. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  5. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  6. 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 116
  7. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 115
  8. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 59
  9. 2021Learnable Companding Quantization for Accurate Low-bit Neural Networkscited by 40
  10. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 38
  11. 2018Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)cited by 19
  12. 2022Accelerating Deep Learning Model Inference on Arm CPUs with Ultra-Low Bit Quantization and Runtimecited by 6
2 more
  1. 2017High performance ultra-low-precision convolutions on mobile devicescited by 5
  2. 2021Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 3; 2026: 3.

8 papers.

  1. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 47
  2. 2025Bitnet.cpp: Efficient Edge Inference for Ternary LLMscited by 17
  3. 2025Characterizing State Space Model and Hybrid Language Model Performance with Long Contextcited by 1
  4. 2025Pushing the Envelope of LLM Inference with Ultra-Low-Bit Quantized Modelscited by 1
  5. 2026SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Modelscited by 1
  6. 2026Breaking the 1.58-bit Barrier for Ternary LLMscited by 1
  7. 2023DeepliteRT: Computer Vision at the Edgecited by 0
  8. 2026CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Expertscited by 0