# ChannelAugment: Improving generalization of multi-channel ASR by training with input channel randomization

arXiv: https://arxiv.org/abs/2109.11225 · HTML: https://mlsanity.com/paper/2109.11225/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 18 of 18)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [SpecAugment on Large Scale Datasets](https://mlsanity.com/paper/1912.05533/) (2019)
- [MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition](https://mlsanity.com/paper/1910.06522/) (2019)
- [Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard](https://mlsanity.com/paper/2001.07263/) (2020)
- [Improving sequence-to-sequence speech recognition training with on-the-fly data augmentation](https://mlsanity.com/paper/1910.13296/) (2019)
- [Semi-Supervised Learning with Data Augmentation for End-to-End ASR](https://mlsanity.com/paper/2007.13876/) (2020)
- [Frequency Domain Multi-channel Acoustic Modeling for Distant Speech Recognition](https://mlsanity.com/paper/1903.05299/) (2019)
- [Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR](https://mlsanity.com/paper/1907.04916/) (2019)
- [Unsupervised training of neural mask-based beamforming](https://mlsanity.com/paper/1904.01578/) (2019)
- [Multi-Geometry Spatial Acoustic Modeling for Distant Speech Recognition](https://mlsanity.com/paper/1903.06539/) (2019)
- [Sound Event Localization and Detection Using Activity-Coupled Cartesian DOA Vector and RD3net](https://mlsanity.com/paper/2006.12014/) (2020)
- [Robust Multi-channel Speech Recognition using Frequency Aligned Network](https://mlsanity.com/paper/2002.02520/) (2020)
- [CNN-based MultiChannel End-to-End Speech Recognition for everyday home environments](https://mlsanity.com/paper/1811.02735/) (2018)

## Abstract (arXiv)

End-to-end (E2E) multi-channel ASR systems show state-of-the-art performance in far-field ASR tasks by joint training of a multi-channel front-end along with the ASR model. The main limitation of such systems is that they are usually trained with data from a fixed array geometry, which can lead to degradation in accuracy when a different array is used in testing. This makes it challenging to deploy these systems in practice, as it is costly to retrain and deploy different models for various array configurations. To address this, we present a simple and effective data augmentation technique, which is based on randomly dropping channels in the multi-channel audio input during training, in order to improve the robustness to various array configurations at test time. We call this technique ChannelAugment, in contrast to SpecAugment (SA) which drops time and/or frequency components of a single channel input audio. We apply ChannelAugment to the Spatial Filtering (SF) and Minimum Variance Distortionless Response (MVDR) neural beamforming approaches. For SF, we observe 10.6% WER improvement across various array configurations employing different numbers of microphones. For MVDR, we achieve a 74% reduction in training time without causing degradation of recognition accuracy.