mlsanityData to 7 Oct 2026

Trained Ternary Quantization

arXiv 1612.01064 ↗First posted 4 Dec 2016cs.LGcited by 203

Not on the map. Why?

Abstract (arXiv)

Deep neural networks are widely used in machine learning applications. However, the deployment of large neural networks models can be difficult to deploy on mobile devices with limited power budgets. To solve this problem, we propose Trained Ternary Quantization (TTQ), a method that can reduce the precision of weights in neural networks to ternary values. This method has very little accuracy degradation and can even improve the accuracy of some models (32, 44, 56-layer ResNet) on CIFAR-10 and AlexNet on ImageNet. And our AlexNet model is trained from scratch, which means it's as easy as to train normal full precision model. We highlight our trained quantization method that can learn both ternary values and ternary assignment. During inference, only ternary values (2-bit weights) and scaling factors are needed, therefore our models are nearly 16x smaller than full-precision models. Our ternary models can also be viewed as sparse binary weight networks, which can potentially be accelerated with custom circuit. Experiments on CIFAR-10 show that the ternary models obtained by trained quantization method outperform full-precision models of ResNet-32,44,56 by 0.04%, 0.16%, 0.36%, respectively. On ImageNet, our model outperforms full-precision AlexNet model by 0.3% of Top-1 accuracy and outperforms previous ternary models by 3%.

Key facts. Cited by 203 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 75; 2022: 41; 2023: 32; 2024: 22; 2025: 18; 2026: 15.

203 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  4. 2021Post-Training Quantization for Vision Transformercited by 274
  5. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  6. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  7. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  8. 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 87
  9. 2021Network Quantization with Element-wise Gradient Scalingcited by 82
  10. 2023Stable and low-precision training for large-scale vision-language modelscited by 74
  11. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  12. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
38 more
  1. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  2. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
  3. 2022Masked Wavelet Representation for Compact Neural Radiance Fieldscited by 57
  4. 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
  5. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  6. 2021Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directionscited by 41
  7. 2022Structural Pruning via Latency-Saliency Knapsackcited by 40
  8. 2024Tiny Machine Learning: Progress and Futurescited by 40
  9. 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
  10. 2021When to Prune? A Policy towards Early Structural Pruningcited by 38
  11. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  12. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Modelcited by 37
  13. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  14. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  15. 2021FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Roundingcited by 31
  16. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26
  17. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
  18. 2023Cuttlefish: Low-Rank Model Training without All the Tuningcited by 25
  19. 2021Network Augmentation for Tiny Deep Learningcited by 23
  20. 2023Q-DETR: An Efficient Low-Bit Quantized Detection Transformercited by 23
  21. 2022Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networkscited by 22
  22. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selectioncited by 20
  23. 2023Token-Scaled Logit Distillation for Ternary Weight Generative Language Modelscited by 20
  24. 2024Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scalecited by 20
  25. 2021CPT: Efficient Deep Neural Network Training via Cyclic Precisioncited by 19
  26. 2021Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolutioncited by 19
  27. 2021Distance-aware Quantizationcited by 19
  28. 2022DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networkscited by 19
  29. 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
  30. 2023Learning Long Sequences in Spiking Neural Networkscited by 19
  31. 2021Towards Efficient Graph Convolutional Networks for Point Cloud Handlingcited by 18
  32. 2021TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devicecited by 17
  33. 2023Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learningcited by 17
  34. 2022Lite Pose: Efficient Architecture Design for 2D Human Pose Estimationcited by 16
  35. 2021Generative Zero-shot Network Quantizationcited by 15
  36. 2021ECQx: Explainability-Driven Quantization for Low-Bit and Sparse DNNscited by 15
  37. 2021Convolutional Neural Network Compression through Generalized Kronecker Product Decompositioncited by 14
  38. 2023Holistic Network Virtualization and Pervasive Network Intelligence for 6Gcited by 14

50 most cited of 203 All 203 (JSON)