mlsanityData to 7 Oct 2026

Improving Deliberation by Text-Only and Semi-Supervised Training

arXiv 2206.14716 ↗First posted 29 Jun 2022cs.CL cs.SD eess.AScited by 10

Not on the map. Why?

Abstract (arXiv)

Text-only and semi-supervised training based on audio-only data has gained popularity recently due to the wide availability of unlabeled text and speech data. In this work, we propose incorporating text-only and semi-supervised training into an attention-based deliberation model. By incorporating text-only data in training a bidirectional encoder representation from transformer (BERT) for the deliberation text encoder, and large-scale text-to-speech and audio-only utterances using joint acoustic and text decoder (JATD) and semi-supervised training, we achieved 4%-12% WER reduction for various tasks compared to the baseline deliberation. Compared to a state-of-the-art language model (LM) rescoring method, the deliberation model reduces the Google Voice Search WER by 11% relative. We show that the deliberation model also achieves a positive human side-by-side evaluation compared to the state-of-the-art LM rescorer with reasonable endpointer latencies.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 29 Jun 2022.

Builds on

29 of 42 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  5. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  6. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  7. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  8. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  9. 2021SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Trainingcited by 69
  10. 2019Transformer-Transducer: End-to-End Speech Recognition with Self-Attentioncited by 68
  11. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 68
  12. 2019A spelling correction model for end-to-end speech recognitioncited by 63
17 more
  1. 2020Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capabilitycited by 63
  2. 2019Two-Pass End-to-End Speech Recognitioncited by 62
  3. 2020Deliberation Model Based Two-Pass End-to-End Speech Recognitioncited by 57
  4. 2019Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Modelcited by 52
  5. 2020A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Taskscited by 51
  6. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 51
  7. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  8. 2019A Comparative Study on End-to-end Speech to Text Translationcited by 40
  9. 2021Tied & Reduced RNN-T Decodercited by 35
  10. 2021Transformer Based Deliberation for Two-Pass Speech Recognitioncited by 29
  11. 2021Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learningcited by 26
  12. 2020Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpuscited by 24
  13. 2020Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Convertercited by 16
  14. 2013Feature Learning in Deep Neural Networks - Studies on Speech Recognition Taskscited by 14
  15. 2020Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Datacited by 14
  16. 2021Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Modelcited by 14
  17. 2018Pretraining by Backtranslation for End-to-end ASR in Low-Resource Settingscited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 4; 2025: 2.

10 papers.

  1. 2024Large Language Models are Efficient Learners of Noise-Robust Speech Recognitioncited by 30
  2. 2023Large-scale Language Model Rescoring on Long-form Datacited by 14
  3. 2023Massively Multilingual Shallow Fusion with Large Language Modelscited by 10
  4. 2022Scaling Up Deliberation for Multilingual ASRcited by 6
  5. 2024Transformer-based Model for ASR N-Best Rescoring and Rewritingcited by 3
  6. 2024Full-text Error Correction for Chinese Speech Recognition with Large Language Modelcited by 3
  7. 2023Masked Audio Text Encoders are Effective Multi-Modal Rescorerscited by 2
  8. 2024Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Modelscited by 0
  9. 2025Chain of Correction for Full-text Speech Recognition with Large Language Modelscited by 0
  10. 2025Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correctioncited by 0