mlsanityData to 14 Oct 2026

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers

arXiv 2310.17723 ↗First posted 26 Oct 2023cs.LG cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Quantization techniques are pivotal in reducing the memory and computational demands of deep neural network inference. Existing solutions, such as ZeroQuant, offer dynamic quantization for models like BERT and GPT but overlook crucial memory-bounded operators and the complexities of per-token quantization. Addressing these gaps, we present a novel, fully hardware-enhanced robust optimized post-training W8A8 quantization framework, ZeroQuant-HERO. This framework uniquely integrates both memory bandwidth and compute-intensive operators, aiming for optimal hardware performance. Additionally, it offers flexibility by allowing specific INT8 modules to switch to FP16/BF16 mode, enhancing accuracy.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 25 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

22 of 25 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  6. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  7. 2019Learned Step Size Quantizationcited by 628
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  9. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  10. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  11. 2019Q8BERT: Quantized 8Bit BERTcited by 300
  12. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
10 more
  1. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  2. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  3. 2020TernaryBERT: Distillation-aware Ultra-low Bit BERTcited by 148
  4. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  5. 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 106
  6. 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
  7. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 79
  8. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 60
  9. 2023ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formatscited by 44
  10. 2021KDLSQ-BERT: A Quantized Bert Combining Knowledge Distillation with Learned Step Size Quantizationcited by 15

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Taskscited by 14

Similar, not cited

Close in wording, no citation either way

  1. 2025Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference
  2. 2021Integer-only Zero-shot Quantization for Efficient Speech Recognition
  3. 2025End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
  4. 2022Sub-8-Bit Quantization Aware Training for 8-Bit Neural Network Accelerator with On-Device Speech Recognition
  5. 2022QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
  6. 2024Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
  7. 2023SONIQ: System-Optimized Noise-Injected Ultra-Low-Precision Quantization with Full-Precision Parity
  8. 2025RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping