# Very Deep Multilingual Convolutional Neural Networks for LVCSR

arXiv: https://arxiv.org/abs/1509.08967 · HTML: https://mlsanity.com/paper/1509.08967/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2015.

## Built on by (most cited first, 11 of 11)

- [MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks](https://mlsanity.com/paper/2103.06132/) (2021)
- [MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognition](https://mlsanity.com/paper/2302.13750/) (2023)
- [Learning strides in convolutional neural networks](https://mlsanity.com/paper/2202.01653/) (2022)
- [XLST: Cross-lingual Self-training to Learn Multilingual Representation for Low Resource Speech Recognition](https://mlsanity.com/paper/2103.08207/) (2021)
- [Distilling a Pretrained Language Model to a Multilingual ASR Model](https://mlsanity.com/paper/2206.12638/) (2022)
- [DNArch: Learning Convolutional Neural Architectures by Backpropagation](https://mlsanity.com/paper/2302.05400/) (2023)
- [Knowledge Distillation Leveraging Alternative Soft Targets from Non-Parallel Qualified Speech Data](https://mlsanity.com/paper/2112.08878/) (2021)
- [Introducing topography in convolutional neural networks](https://mlsanity.com/paper/2211.13152/) (2022)
- [Adaptive Activation Network For Low Resource Multilingual Speech Recognition](https://mlsanity.com/paper/2205.14326/) (2022)
- [Analysis and Comparison of Two-Level KFAC Methods for Training Deep Neural Networks](https://mlsanity.com/paper/2303.18083/) (2023)
- [Shrinkage Initialization for Smooth Learning of Neural Networks](https://mlsanity.com/paper/2504.09107/) (2025)

## Abstract (arXiv)

Convolutional neural networks (CNNs) are a standard component of many current state-of-the-art Large Vocabulary Continuous Speech Recognition (LVCSR) systems. However, CNNs in LVCSR have not kept pace with recent advances in other domains where deeper neural networks provide superior performance. In this paper we propose a number of architectural advances in CNNs for LVCSR. First, we introduce a very deep convolutional network architecture with up to 14 weight layers. There are multiple convolutional layers before each pooling layer, with small 3x3 kernels, inspired by the VGG Imagenet 2014 architecture. Then, we introduce multilingual CNNs with multiple untied layers. Finally, we introduce multi-scale input features aimed at exploiting more context at negligible computational cost. We evaluate the improvements first on a Babel task for low resource speech recognition, obtaining an absolute 5.77% WER improvement over the baseline PLP DNN by training our CNN on the combined data of six different languages. We then evaluate the very deep CNNs on the Hub5'00 benchmark (using the 262 hours of SWB-1 training data) achieving a word error rate of 11.8% after cross-entropy training, a 1.4% WER improvement (10.6% relative) over the best published CNN result so far.