# End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation arXiv: https://arxiv.org/abs/2204.00540 ยท HTML: https://mlsanity.com/paper/2204.00540/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 37 references that are in the map. - First posted on arXiv 1 Apr 2022. ## Builds on (most cited first, 23 of 23) - [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020) - [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021) - [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019) - [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013) - [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018) - [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019) - [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021) - [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018) - [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021) - [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014) - [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021) - [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016) - [Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2010.10504/) (2020) - [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019) - [TasNet: time-domain audio separation network for real-time, single-channel speech separation](https://mlsanity.com/paper/1711.00541/) (2017) - [An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.04590/) (2021) - [How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR](https://mlsanity.com/paper/2201.06685/) (2022) - [Multichannel End-to-end Speech Recognition](https://mlsanity.com/paper/1703.04783/) (2017) - [Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baseline](https://mlsanity.com/paper/1803.10109/) (2018) - [Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition](https://mlsanity.com/paper/2110.04934/) (2021) - [Efficient Implementation of the Room Simulator for Training Deep Neural Network Acoustic Models](https://mlsanity.com/paper/1712.03439/) (2017) - [Closing the Gap Between Time-Domain Multi-Channel Speech Enhancement on Real and Simulation Conditions](https://mlsanity.com/paper/2110.14139/) (2021) - [A Conformer Based Acoustic Model for Robust Automatic Speech Recognition](https://mlsanity.com/paper/2203.00725/) (2022) ## Built on by (most cited first, 17 of 17) - [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023) - [The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios](https://mlsanity.com/paper/2306.13734/) (2023) - [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023) - [Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR](https://mlsanity.com/paper/2304.04974/) (2023) - [Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognition](https://mlsanity.com/paper/2409.00815/) (2024) - [Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapters](https://mlsanity.com/paper/2401.05111/) (2024) - [Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Experts](https://mlsanity.com/paper/2211.06493/) (2022) - [Breaking the Transcription Bottleneck: Fine-tuning ASR Models for Extremely Low-Resource Fieldwork Languages](https://mlsanity.com/paper/2506.17459/) (2025) - [CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR](https://mlsanity.com/paper/2502.20040/) (2025) - [Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges](https://mlsanity.com/paper/2507.18161/) (2025) - [Investigation for Relative Voice Impression Estimation](https://mlsanity.com/paper/2602.14172/) (2026) - [Exploring the Integration of Speech Separation and Recognition with Self-Supervised Learning Representation](https://mlsanity.com/paper/2307.12231/) (2023) - [Cascaded Multi-task Adaptive Learning Based on Neural Architecture Search](https://mlsanity.com/paper/2310.17664/) (2023) - [Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representations](https://mlsanity.com/paper/2601.21084/) (2026) - [Frontend Token Enhancement for Token-Based Speech Recognition](https://mlsanity.com/paper/2602.04217/) (2026) - [Joint Enhancement and Classification using Coupled Diffusion Models of Signals and Logits](https://mlsanity.com/paper/2602.15405/) (2026) - [Transcript-Supervised Post-Training of Generative Speech Enhancement on Real Recordings via Reinforce Adjoint Matching](https://mlsanity.com/paper/2609.29405/) (2026) ## Abstract (arXiv) This work presents our end-to-end (E2E) automatic speech recognition (ASR) model targetting at robust speech recognition, called Integraded speech Recognition with enhanced speech Input for Self-supervised learning representation (IRIS). Compared with conventional E2E ASR models, the proposed E2E model integrates two important modules including a speech enhancement (SE) module and a self-supervised learning representation (SSLR) module. The SE module enhances the noisy speech. Then the SSLR module extracts features from enhanced speech to be used for speech recognition (ASR). To train the proposed model, we establish an efficient learning scheme. Evaluation results on the monaural CHiME-4 task show that the IRIS model achieves the best performance reported in the literature for the single-channel CHiME-4 benchmark (2.0% for the real development and 3.9% for the real test) thanks to the powerful pre-trained SSLR module and the fine-tuned SE module.