mlsanityData to 7 Oct 2026

Text-Only Domain Adaptation for End-to-End Speech Recognition through Down-Sampling Acoustic Representation

arXiv 2309.02459 ↗First posted 4 Sep 2023cs.SD cs.CL eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Mapping two modalities, speech and text, into a shared representation space, is a research topic of using text-only data to improve end-to-end automatic speech recognition (ASR) performance in new domains. However, the length of speech representation and text representation is inconsistent. Although the previous method up-samples the text representation to align with acoustic modality, it may not match the expected actual duration. In this paper, we proposed novel representations match strategy through down-sampling acoustic representation to align with text modality. By introducing a continuous integrate-and-fire (CIF) module generating acoustic representations consistent with token length, our ASR model can learn unified representations from both modalities better, allowing for domain adaptation using text-only data of the target domain. Experiment results of new domain data demonstrate the effectiveness of the proposed method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 30 references that are in the map. First posted on arXiv 4 Sep 2023.

Builds on

26 of 30 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  5. 2015Listen, Attend and Spellcited by 542
  6. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  7. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 189
  8. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  9. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 125
  10. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 123
  11. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
  12. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
14 more
  1. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  2. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  3. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 76
  4. 2014End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Resultscited by 68
  5. 2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
  6. 2021Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Modelscited by 25
  7. 2021Injecting Text in Self-Supervised Speech Pretrainingcited by 24
  8. 2022JOIST: A Joint Speech and Text Streaming Model For ASRcited by 24
  9. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
  10. 2020Multitask Training with Text Data for End-to-End Speech Recognitioncited by 19
  11. 2021Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Modelcited by 14
  12. 2018Exploring RNN-Transducer for Chinese Speech Recognitioncited by 8
  13. 2022Improving Mandarin Speech Recogntion with Block-augmented Transformercited by 6
  14. 2022Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognitioncited by 5

Built on by

Not cited yet.