mlsanityData to 7 Oct 2026

Zero-Shot Dynamic Quantization for Transformer Inference

arXiv 2211.09744 ↗First posted 17 Nov 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a novel run-time method for significantly reducing the accuracy loss associated with quantizing BERT-like models to 8-bit integers. Existing methods for quantizing models either modify the training procedure,or they require an additional calibration step to adjust parameters that also requires a selected held-out dataset. Our method permits taking advantage of quantization without the need for these adjustments. We present results on several NLP tasks demonstrating the usefulness of this technique.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 17 Nov 2022.

Builds on

24 of 51 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  10. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  11. 2018Neural Network Acceptability Judgmentscited by 952
  12. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
12 more
  1. 2017Convolutional Sequence to Sequence Learningcited by 550
  2. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  3. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  5. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  6. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  7. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  8. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  9. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  10. 2019Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Modelcited by 58
  11. 2019A BERT Baseline for the Natural Questionscited by 33
  12. 2018Pieces of Eight: 8-bit Neural Machine Translationcited by 7

Built on by

Not cited yet.