mlsanityData to 14 Oct 2026

Enhancing Fully Formatted End-to-End Speech Recognition with Knowledge Distillation via Multi-Codebook Vector Quantization

arXiv 2512.18967 ↗First posted 22 Dec 2025eess.AScited by 0

On the map: Knowledge distillation → Speech and audio distillation

Abstract (arXiv)

Conventional automatic speech recognition (ASR) models typically produce outputs as normalized texts lacking punctuation and capitalization, necessitating post-processing models to enhance readability. This approach, however, introduces additional complexity and latency due to the cascaded system design. In response to this challenge, there is a growing trend to develop end-to-end (E2E) ASR models capable of directly predicting punctuation and capitalization, though this area remains underexplored. In this paper, we propose an enhanced fully formatted E2E ASR model that leverages knowledge distillation (KD) through multi-codebook vector quantization (MVQ). Experimental results demonstrate that our model significantly outperforms previous works in word error rate (WER) both with and without punctuation and capitalization, and in punctuation error rate (PER). Evaluations on the LibriSpeech-PC test-clean and test-other subsets show that our model achieves state-of-the-art results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation. Builds on 20 of its 33 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

20 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Predicting Multi-Codebook Vector Quantization Indexes for Knowledge Distillation31 Oct
  2. 2021Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models7 Oct
  3. 2021CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognition14 Jun

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  6. 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 151
  7. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  8. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
  9. 2020Efficient Knowledge Distillation for RNN-Transducer Modelscited by 38
  10. 2023PromptASR for contextualized ASR with controllable stylecited by 19
  11. 2020Multimodal Semi-supervised Learning Framework for Punctuation Prediction in Conversational Speechcited by 18
  12. 2021Capitalization and Punctuation Restoration: a Surveycited by 13
5 more
  1. 2023Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech Recognitioncited by 12
  2. 2022End-to-end Speech-to-Punctuated-Text Recognitioncited by 8
  3. 2021Joint prediction of truecasing and punctuation for conversational speech in low-resource scenarioscited by 6
  4. 2020Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?cited by 4
  5. 2020End to End ASR System with Automatic Punctuation Insertioncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Leave No Knowledge Behind During Knowledge Distillation: Towards Practical and Effective Knowledge Distillation for Code-Switching ASR Using Realistic Data
  2. 2023Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation
  3. 2023Knowledge Distillation from Multiple Foundation Models for End-to-End Speech Recognition
  4. 2024Joint Optimization of Streaming and Non-Streaming Automatic Speech Recognition with Multi-Decoder and Knowledge Distillation
  5. 2024Optimizing Byte-level Representation for End-to-end ASR
  6. 2021Hierarchical Transformer-based Large-Context End-to-end ASR with Large-Context Knowledge Distillation
  7. 2023DistillW2V2: A Small and Streaming Wav2vec 2.0 Based ASR Model
  8. 2023On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition