mlsanityData to 7 Oct 2026

Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations

arXiv 1609.07061 ↗First posted 22 Sep 2016cs.NE cs.LGcited by 413

Not on the map. Why?

Abstract (arXiv)

We introduce a method to train Quantized Neural Networks (QNNs) --- neural networks with extremely low precision (e.g., 1-bit) weights and activations, at run-time. At train-time the quantized weights and activations are used for computing the parameter gradients. During the forward pass, QNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations. As a result, power consumption is expected to be drastically reduced. We trained QNNs over the MNIST, CIFAR-10, SVHN and ImageNet datasets. The resulting QNNs achieve prediction accuracy comparable to their 32-bit counterparts. For example, our quantized version of AlexNet with 1-bit weights and 2-bit activations achieves 51% top-1 accuracy. Moreover, we quantize the parameter gradients to 6-bits as well which enables gradients computation using only bit-wise operation. Quantized recurrent neural networks were tested over the Penn Treebank dataset, and achieved comparable accuracy as their 32-bit counterparts using only 4-bits. Last but not least, we programmed a binary matrix multiplication GPU kernel with which it is possible to run our MNIST QNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The QNN code is available online.

Key facts. Cited by 413 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 102; 2022: 60; 2023: 62; 2024: 73; 2025: 81; 2026: 35.

413 papers.

Sort:

By area:

  1. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  2. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  3. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  4. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  5. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  6. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  7. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  8. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  9. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  10. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  11. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  12. 2022Machine Learning Operations (MLOps): Overview, Definition, and Architecturecited by 126
38 more
  1. 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 125
  2. 2021Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approachcited by 105
  3. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  4. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  5. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  6. 2021SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)cited by 67
  7. 2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
  8. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  9. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
  10. 2021hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devicescited by 61
  11. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  12. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  13. 2021A Survey on Green Deep Learningcited by 52
  14. 2022BiT: Robustly Binarized Multi-distilled Transformercited by 51
  15. 2023Intriguing Properties of Quantization at Scalecited by 48
  16. 2024Training of Physical Neural Networkscited by 47
  17. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 46
  18. 2021Hessian-Aware Pruning and Optimal Neural Implantcited by 43
  19. 2021UnitedQA: A Hybrid Approach for Open Domain Question Answeringcited by 42
  20. 2021Fast convolutional neural networks on FPGAs with hls4mlcited by 42
  21. 2021Instance-Conditional Knowledge Distillation for Object Detectioncited by 42
  22. 2023Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based acceleratorscited by 42
  23. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  24. 2021Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directionscited by 41
  25. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 38
  26. 2023A Survey on Efficient Training of Transformerscited by 37
  27. 2024Low-Rank Adaptation for Foundation Models: A Comprehensive Reviewcited by 35
  28. 2022Rare Gems: Finding Lottery Tickets at Initializationcited by 34
  29. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  30. 2021FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Roundingcited by 31
  31. 2021Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devicescited by 30
  32. 2022Fine-tuning Language Models over Slow Networks using Activation Compression with Guaranteescited by 29
  33. 2021Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operatorscited by 28
  34. 2022Training Your Sparse Neural Network Better with Any Maskcited by 26
  35. 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 26
  36. 2024Selective Attention Improves Transformercited by 26
  37. 2023Cuttlefish: Low-Rank Model Training without All the Tuningcited by 25
  38. 2023The Efficiency Spectrum of Large Language Models: An Algorithmic Surveycited by 25

50 most cited of 413 All 413 (JSON)