mlsanityData to 14 Oct 2026

Sub-8-Bit Quantization Aware Training for 8-Bit Neural Network Accelerator with On-Device Speech Recognition

arXiv 2206.15408 ↗First posted 30 Jun 2022eess.AS cs.AI eess.SPcited by 5

Not on the map. Why?

Abstract (arXiv)

We present a novel sub-8-bit quantization-aware training (S8BQAT) scheme for 8-bit neural network accelerators. Our method is inspired from Lloyd-Max compression theory with practical adaptations for a feasible computational overhead during training. With the quantization centroids derived from a 32-bit baseline, we augment training loss with a Multi-Regional Absolute Cosine (MRACos) regularizer that aggregates weights towards their nearest centroid, effectively acting as a pseudo compressor. Additionally, a periodically invoked hard compressor is introduced to improve the convergence rate by emulating runtime model weight quantization. We apply S8BQAT on speech recognition tasks using Recurrent Neural NetworkTransducer (RNN-T) architecture. With S8BQAT, we are able to increase the model parameter size to reduce the word error rate by 4-16% relatively, while still improving latency by 5%.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 19 of its 27 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

19 of 27 references.

References · 19

  1. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  3. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  4. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  5. 2019On the Efficacy of Knowledge Distillationcited by 381
  6. 2019Q8BERT: Quantized 8Bit BERTcited by 300
  7. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 100
  8. 2019AutoQ: Automated Kernel-Wise Neural Network Quantizationcited by 63
  9. 2019Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Modelcited by 59
  10. 2020End-to-End Neural Transformer Based Spoken Language Understandingcited by 31
  11. 2020Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Inputcited by 24
  12. 2021PQK: Model Compression via Pruning, Quantization, and Knowledge Distillationcited by 21
7 more
  1. 2021End-to-End Multi-Channel Transformer for Speech Recognitioncited by 17
  2. 2021Post-Training Sparsity-Aware Quantizationcited by 17
  3. 2019Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Codingcited by 15
  4. 2021Sparsification via Compressed Sensing for Automatic Speech Recognitioncited by 8
  5. 2021Weight, Block or Unit? Exploring Sparsity Tradeoffs for Speech Enhancement on Tiny Neural Acceleratorscited by 8
  6. 2020Source-Aware Neural Speech Coding for Noisy Speech Compressioncited by 5
  7. 2019Incremental Binarization On Recurrent Neural Networks For Single-Channel Source Separationcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1; 2025: 1.

5 papers.

  1. 2023A Model for Every User and Budget: Label-Free and Personalized Mixed-Precision Quantizationcited by 8
  2. 2022Sub-8-bit quantization for on-device speech recognition: a regularization-free approachcited by 4
  3. 2024Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Modelscited by 3
  4. 2025BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexingcited by 2
  5. 2023Enhancing Quantised End-to-End ASR Models via Personalisationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Accelerator-Aware Training for Transducer-Based Speech Recognition
  2. 2021MOHAQ: Multi-Objective Hardware-Aware Quantization of Recurrent Neural Networks
  3. 2022Sub 8-Bit Quantization of Streaming Keyword Spotting Models for Embedded Chipsets
  4. 2021Integer-only Zero-shot Quantization for Efficient Speech Recognition
  5. 20224-bit Conformer with Native Quantization Aware Training for Speech Recognition
  6. 2021A White Paper on Neural Network Quantization
  7. 2022Accelerating Inference and Language Model Fusion of Recurrent Neural Network Transducers via End-to-End 4-bit Quantization
  8. 2026Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures