Data to 7 Oct 2026

Ternary Weight Networks

arXiv 1605.04711 ↗First posted 16 May 2016cs.CVcited by 195

Not on the map. Why?

Abstract (arXiv)

We present a memory and computation efficient ternary weight networks (TWNs) - with weights constrained to +1, 0 and -1. The Euclidian distance between full (float or double) precision weights and the ternary weights along with a scaling factor is minimized in training stage. Besides, a threshold-based ternary function is optimized to get an approximated solution which can be fast and easily computed. TWNs have shown better expressive abilities than binary precision counterparts. Meanwhile, TWNs achieve up to 16× model compression rate and need fewer multiplications compared with the float32 precision counterparts. Extensive experiments on MNIST, CIFAR-10, and ImageNet datasets show that the TWNs achieve much better result than the Binary-Weight-Networks (BWNs) and the classification performance on MNIST and CIFAR-10 is very close to the full precision networks. We also verify our method on object detection task and show that TWNs significantly outperforms BWN by more than 10% mAP on PASCAL VOC dataset. The pytorch version of source code is available at: https://github.com/Thinklab-SJTU/twns.

Key facts. Cited by 195 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 61; 2022: 42; 2023: 23; 2024: 27; 2025: 23; 2026: 19.

195 papers.

Sort:

By area:

2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
2021I-BERT: Integer-only BERT Quantizationcited by 250
2023EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingScited by 191
2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
2021Network Quantization with Element-wise Gradient Scalingcited by 82
2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
2021hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devicescited by 61
2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
2021Differentiable Model Compression via Pseudo Quantization Noisecited by 44

38 more

2022AdaBin: Improving Binary Neural Networks with Adaptive Binary Setscited by 38
2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Modelcited by 37
2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
2021Distribution Adaptive INT8 Quantization for Training CNNscited by 27
2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 26
2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
2021An Information-Theoretic Justification for Model Pruningcited by 23
20213D Scene Compression through Entropy Penalized Neural Representation Functionscited by 23
2022DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantizationcited by 23
2023Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Casescited by 23
2022Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networkscited by 22
2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Modelscited by 22
2024Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scalecited by 20
2021CPT: Efficient Deep Neural Network Training via Cyclic Precisioncited by 19
2021Charged particle tracking via edge-classifying interaction networkscited by 19
2021Distance-aware Quantizationcited by 19
2022Extreme Compression for Pre-trained Transformers Made Simple and Efficientcited by 19
2023RobustMQ: Benchmarking Robustness of Quantized Modelscited by 19
2021Generative Zero-shot Network Quantizationcited by 15
2023AutoFHE: Automated Adaption of CNNs for Efficient Evaluation over FHEcited by 15
2024TernaryLLM: Ternarized Large Language Modelcited by 15
2021Convolutional Neural Network Compression through Generalized Kronecker Product Decompositioncited by 14
2021Quantization of Deep Neural Networks for Accurate Edge Computingcited by 13
2021Communication-Efficient Federated Learning with Binary Neural Networkscited by 13
2021Neural Network Quantization for Efficient Inference: A Surveycited by 13
2021Prune Your Model Before Distill Itcited by 12
2022HCFL: A High Compression Approach for Communication-Efficient Federated Learning in Very Large Scale IoT Networkscited by 12
2023Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theorycited by 12
2021Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Losscited by 11
2021Demystifying and Generalizing BinaryConnectcited by 11
2022Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantizationcited by 11
2024KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questionscited by 11
2021Mixed Precision of Quantization of Transformer Language Models for Speech Recognitioncited by 10
2021BMPQ: Bit-Gradient Sensitivity Driven Mixed-Precision Quantization of DNNs from Scratchcited by 10
2022TerViT: An Efficient Ternary Vision Transformercited by 10
2022FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Supportcited by 10

50 most cited of 195 All 195 (JSON)