mlsanityData to 14 Oct 2026

Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR

arXiv 2409.08797 ↗First posted 13 Sep 2024cs.CL cs.SD eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable. In this paper, SSL discrete speech features extracted from WavLM models are used as additional cross-utterance acoustic context features in Zipformer-Transducer ASR systems. The efficacy of replacing Fbank features with discrete token features for modelling either cross-utterance contexts (from preceding and future segments), or current utterance's internal contexts alone, or both at the same time, are demonstrated thoroughly on the Gigaspeech 1000-hr corpus. The best Zipformer-Transducer system using discrete tokens based cross-utterance context features outperforms the baseline using utterance internal context only with statistically significant word error rate (WER) reductions of 0.32% to 0.41% absolute (2.78% to 3.54% relative) on the dev and test data. The lowest published WER of 11.15% and 11.14% were obtained on the dev and test sets. Our work is open-source and publicly available at https://github.com/open-creator/icefall/tree/master/egs/gigaspeech/ContextASR.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 13 Sep 2024.

Builds on

36 of 51 references.

References · 36

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2020Longformer: The Long-Document Transformercited by 3,413
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  6. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  7. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  8. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
  9. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  10. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  11. 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 114
  12. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 97
24 more
  1. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  2. 2019Effectiveness of self-supervised pre-training for speech recognitioncited by 72
  3. 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
  4. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  5. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 32
  6. 2016Attentive Pooling Networkscited by 28
  7. 2023Exploration of Efficient End-to-End ASR using Discretized Input from Self-Supervised Learningcited by 28
  8. 2018Dialog-context aware end-to-end speech recognitioncited by 27
  9. 2021Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformerscited by 25
  10. 2023Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTScited by 24
  11. 2021Transformer Language Models with LSTM-based Cross-utterance Information Representationcited by 15
  12. 2023Acoustic BPE for Speech Generation with Discrete Tokenscited by 15
  13. 2024The Interspeech 2024 Challenge on Speech Processing Using Discrete Unitscited by 12
  14. 2020Contextualizing ASR Lattice Rescoring with Hybrid Pointer Network Language Modelcited by 11
  15. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  16. 2024Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Modelcited by 8
  17. 2022Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASRcited by 7
  18. 2022LongFNT: Long-form Speech Recognition with Factorized Neural Transducercited by 6
  19. 2023Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systemscited by 6
  20. 2024High Fidelity Text-to-Speech Via Discrete Tokens Using Token Transducer and Group Masked Language Modelcited by 6
  21. 2020LSTM-LM with Long-Term History for First-Pass Decoding in Conversational Speech Recognitioncited by 4
  22. 2022Improving Transformer-based Conversational ASR by Inter-Sentential Attention Mechanismcited by 4
  23. 2023Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASRcited by 4
  24. 2024Advanced Long-Content Speech Recognition With Factorized Neural Transducercited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  2. 2026Controllable Accent Normalization via Discrete Diffusioncited by 0