mlsanityData to 7 Oct 2026

Q8BERT: Quantized 8Bit BERT

arXiv 1910.06188 ↗First posted 14 Oct 2019cs.CL cs.LGcited by 296

Not on the map. Why?

Abstract (arXiv)

Recently, pre-trained Transformer based language models such as BERT and GPT, have shown great improvement in many Natural Language Processing (NLP) tasks. However, these models contain a large amount of parameters. The emergence of even larger and more accurate models such as GPT2 and Megatron, suggest a trend of large pre-trained Transformer models. However, using these large models in production environments is a complex task requiring a large amount of compute, memory and power resources. In this work we show how to perform quantization-aware training during the fine-tuning phase of BERT in order to compress BERT by 4× with minimal accuracy loss. Furthermore, the produced quantized model can accelerate inference speed if it is optimized for 8bit Integer supporting hardware.

Key facts. Cited by 296 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 61; 2022: 69; 2023: 80; 2024: 47; 2025: 25; 2026: 14.

296 papers.

Sort:

By area:

  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  2. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  3. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  4. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  5. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  6. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  7. 2022Efficiently Scaling Transformer Inferencecited by 529
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  9. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  10. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  11. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  12. 2021Pre-Trained Models: Past, Present and Futurecited by 319
38 more
  1. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  2. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  3. 2021Post-Training Quantization for Vision Transformercited by 274
  4. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  5. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  6. 2023Large Language Models as General Pattern Machinescited by 196
  7. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
  8. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  9. 2023Speculative Decoding with Big Little Decodercited by 163
  10. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  11. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  12. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
  13. 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 149
  14. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  15. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  16. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  17. 2021Learned Token Pruning for Transformerscited by 138
  18. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  19. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  20. 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 110
  21. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 105
  22. 2023Fast Attention Requires Bounded Entriescited by 102
  23. 2021Finetuning Pretrained Transformers into RNNscited by 94
  24. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  25. 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
  26. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  27. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
  28. 2021Vision Transformer Pruningcited by 77
  29. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  30. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 71
  31. 2022BiBERT: Accurate Fully Binarized BERTcited by 70
  32. 2021Prune Once for All: Sparse Pre-Trained Language Modelscited by 66
  33. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
  34. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  35. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  36. 2023Training Transformers with 4-bit Integerscited by 58
  37. 2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruningcited by 56
  38. 2021NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Searchcited by 55

50 most cited of 296 All 296 (JSON)