mlsanityData to 7 Oct 2026

End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation

arXiv 2204.00540 ↗First posted 1 Apr 2022cs.SD cs.CL eess.AScited by 17

Not on the map. Why?

Abstract (arXiv)

This work presents our end-to-end (E2E) automatic speech recognition (ASR) model targetting at robust speech recognition, called Integraded speech Recognition with enhanced speech Input for Self-supervised learning representation (IRIS). Compared with conventional E2E ASR models, the proposed E2E model integrates two important modules including a speech enhancement (SE) module and a self-supervised learning representation (SSLR) module. The SE module enhances the noisy speech. Then the SSLR module extracts features from enhanced speech to be used for speech recognition (ASR). To train the proposed model, we establish an efficient learning scheme. Evaluation results on the monaural CHiME-4 task show that the IRIS model achieves the best performance reported in the literature for the single-channel CHiME-4 benchmark (2.0% for the real development and 3.9% for the real test) thanks to the powerful pre-trained SSLR module and the fine-tuned SE module.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 23 of its 37 references that are in the map. First posted on arXiv 1 Apr 2022.

Builds on

23 of 37 references.

References · 23

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  6. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  7. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  8. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  9. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  10. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  11. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  12. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
11 more
  1. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  2. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  3. 2017TasNet: time-domain audio separation network for real-time, single-channel speech separationcited by 115
  4. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  5. 2022How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASRcited by 27
  6. 2017Multichannel End-to-end Speech Recognitioncited by 17
  7. 2018Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baselinecited by 17
  8. 2021Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognitioncited by 15
  9. 2017Efficient Implementation of the Room Simulator for Training Deep Neural Network Acoustic Modelscited by 9
  10. 2021Closing the Gap Between Time-Domain Multi-Channel Speech Enhancement on Real and Simulation Conditionscited by 6
  11. 2022A Conformer Based Acoustic Model for Robust Automatic Speech Recognitioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 2; 2025: 3; 2026: 5.

17 papers.

Sort:

By area:

  1. 2023End-to-End Speech Recognition: A Surveycited by 127
  2. 2023The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarioscited by 20
  3. 2023One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognitioncited by 12
  4. 2023Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASRcited by 10
  5. 2024Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognitioncited by 7
  6. 2024Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapterscited by 4
  7. 2025Breaking the Transcription Bottleneck: Fine-tuning ASR Models for Extremely Low-Resource Fieldwork Languagescited by 3
  8. 2022Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Expertscited by 2
  9. 2025CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASRcited by 2
  10. 2025Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challengescited by 2
  11. 2026Investigation for Relative Voice Impression Estimationcited by 1
  12. 2023Exploring the Integration of Speech Separation and Recognition with Self-Supervised Learning Representationcited by 0
5 more
  1. 2023Cascaded Multi-task Adaptive Learning Based on Neural Architecture Searchcited by 0
  2. 2026Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representationscited by 0
  3. 2026Frontend Token Enhancement for Token-Based Speech Recognitioncited by 0
  4. 2026Joint Enhancement and Classification using Coupled Diffusion Models of Signals and Logitscited by 0
  5. 2026Transcript-Supervised Post-Training of Generative Speech Enhancement on Real Recordings via Reinforce Adjoint Matchingcited by 0