# Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition

arXiv: https://arxiv.org/abs/2107.02670 · HTML: https://mlsanity.com/paper/2107.02670/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 12 of 12)

- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition](https://mlsanity.com/paper/1910.06522/) (2019)
- [Data Augmenting Contrastive Learning of Speech Representations in the Time Domain](https://mlsanity.com/paper/2007.00991/) (2020)
- [Multichannel End-to-end Speech Recognition](https://mlsanity.com/paper/1703.04783/) (2017)
- [CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency](https://mlsanity.com/paper/2005.13326/) (2020)
- [End-to-End Far-Field Speech Recognition with Unified Dereverberation and Beamforming](https://mlsanity.com/paper/2005.10479/) (2020)
- [Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients](https://mlsanity.com/paper/2011.05649/) (2020)
- [Quaternion Neural Networks for Multi-channel Distant Speech Recognition](https://mlsanity.com/paper/2005.08566/) (2020)

## Abstract (arXiv)

Recently, the end-to-end training approach for neural beamformer-supported multi-channel ASR has shown its effectiveness in multi-channel speech recognition. However, the integration of multiple modules makes it more difficult to perform end-to-end training, particularly given that the multi-channel speech corpus recorded in real environments with a sizeable data scale is relatively limited. This paper explores the usage of single-channel data to improve the multi-channel end-to-end speech recognition system. Specifically, we design three schemes to exploit the single-channel data, namely pre-training, data scheduling, and data simulation. Extensive experiments on CHiME4 and AISHELL-4 datasets demonstrate that all three methods improve the multi-channel end-to-end training stability and speech recognition performance, while the data scheduling approach keeps a much simpler pipeline (vs. pre-training) and less computation cost (vs. data simulation). Moreover, we give a thorough analysis of our systems, including how the performance is affected by the choice of front-end, the data augmentation, training strategy, and single-channel data size.