mlsanityData to 7 Oct 2026

EAT: Enhanced ASR-TTS for Self-supervised Speech Recognition

arXiv 2104.07474 ↗First posted 13 Apr 2021eess.AS cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Self-supervised ASR-TTS models suffer in out-of-domain data conditions. Here we propose an enhanced ASR-TTS (EAT) model that incorporates two main features: 1) The ASRrightarrowTTS direction is equipped with a language model reward to penalize the ASR hypotheses before forwarding it to TTS. 2) In the TTSrightarrowASR direction, a hyper-parameter is introduced to scale the attention context from synthesized speech before sending it to ASR to handle out-of-domain data. Training strategies and the effectiveness of the EAT model are explored under out-of-domain data conditions. The results show that EAT reduces the performance gap between supervised and self-supervised training significantly by absolute 2.6% and 2.7% on Librispeech and BABEL respectively.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 13 Apr 2021.

Builds on

18 of 26 references.

References · 18

  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  5. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  6. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  7. 2019Self-Training for End-to-End Speech Recognitioncited by 128
  8. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  9. 2019Effectiveness of self-supervised pre-training for speech recognitioncited by 72
  10. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  11. 2017Listening while Speaking: Speech Chain by Deep Learningcited by 46
  12. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 46
6 more
  1. 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
  2. 2018Cycle-consistency training for end-to-end speech recognitioncited by 31
  3. 2018Back-Translation-Style Data Augmentation for End-to-End ASRcited by 28
  4. 2020Semi-Supervised Speech Recognition via Local Prior Matchingcited by 13
  5. 2018End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimatorcited by 11
  6. 2019Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Textcited by 10

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 6; 2024: 4.

13 papers.

Sort:

  1. 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
  2. 2023Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generatorcited by 11
  3. 2022Boosting Cross-Domain Speech Recognition with Self-Supervisioncited by 10
  4. 2024Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognitioncited by 9
  5. 2022USTED: Improving ASR with a Unified Speech and Text Encoder-Decodercited by 7
  6. 2023Text Generation with Speech Synthesis for ASR Data Augmentationcited by 5
  7. 2023On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognitioncited by 4
  8. 2024On the Effect of Purely Synthetic Training Data for Different Automatic Speech Recognition Architecturescited by 4
  9. 2024On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognitioncited by 3
  10. 2023SpeeChain: A Speech Toolkit for Large-Scale Machine Speech Chaincited by 2
  11. 2023Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Trackingcited by 1
  12. 2023End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representationscited by 0
1 more
  1. 2024Written Term Detection Improves Spoken Term Detectioncited by 0