# Very Deep Convolutional Networks for End-to-End Speech Recognition

arXiv: https://arxiv.org/abs/1610.03022 · HTML: https://mlsanity.com/paper/1610.03022/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2016.

## Built on by (most cited first, 30 of 34)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network](https://mlsanity.com/paper/2104.02133/) (2021)
- [Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer](https://mlsanity.com/paper/2305.14280/) (2023)
- [Pushing the Limits of Non-Autoregressive Speech Recognition](https://mlsanity.com/paper/2104.03416/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Homogeneous Architecture Augmentation for Neural Predictor](https://mlsanity.com/paper/2107.13153/) (2021)
- [Efficient Weight factorization for Multilingual Speech Recognition](https://mlsanity.com/paper/2105.03010/) (2021)
- [Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models](https://mlsanity.com/paper/2409.18654/) (2024)
- [Network In Graph Neural Network](https://mlsanity.com/paper/2111.11638/) (2021)
- [ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognition](https://mlsanity.com/paper/2209.14868/) (2022)
- [On Compressing Sequences for Self-Supervised Speech Models](https://mlsanity.com/paper/2210.07189/) (2022)
- [Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing](https://mlsanity.com/paper/2211.01522/) (2022)
- [Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition](https://mlsanity.com/paper/2102.07259/) (2021)
- [A Unified Speaker Adaptation Approach for ASR](https://mlsanity.com/paper/2110.08545/) (2021)
- [Insights on Neural Representations for End-to-End Speech Recognition](https://mlsanity.com/paper/2205.09456/) (2022)
- [Sequential Estimation of Gaussian Process-based Deep State-Space Models](https://mlsanity.com/paper/2301.12528/) (2023)
- [A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robots](https://mlsanity.com/paper/2103.08183/) (2021)
- [Do End-to-End Speech Recognition Models Care About Context?](https://mlsanity.com/paper/2102.09928/) (2021)
- [Adversarial Attacks on Speech Recognition Systems for Mission-Critical Applications: A Survey](https://mlsanity.com/paper/2202.10594/) (2022)
- [Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet](https://mlsanity.com/paper/2406.17825/) (2024)
- [Polyphonic pitch detection with convolutional recurrent neural networks](https://mlsanity.com/paper/2202.02115/) (2022)
- [A Complementary Joint Training Approach Using Unpaired Speech and Text for Low-Resource Automatic Speech Recognition](https://mlsanity.com/paper/2204.02023/) (2022)
- [Prostate Cancer Malignancy Detection and localization from mpMRI using auto-Deep Learning: One Step Closer to Clinical Utilization](https://mlsanity.com/paper/2206.06235/) (2022)
- [Improving Semi-supervised End-to-end Automatic Speech Recognition using CycleGAN and Inter-domain Losses](https://mlsanity.com/paper/2210.11642/) (2022)
- [TS-ENAS:Two-Stage Evolution for Cell-based Network Architecture Search](https://mlsanity.com/paper/2310.09525/) (2023)
- [DDC-PIM: Efficient Algorithm/Architecture Co-design for Doubling Data Capacity of SRAM-based Processing-In-Memory](https://mlsanity.com/paper/2310.20424/) (2023)
- [Dynamic Latency for CTC-Based Streaming Automatic Speech Recognition With Emformer](https://mlsanity.com/paper/2203.15613/) (2022)
- [DEFORMER: Coupling Deformed Localized Patterns with Global Context for Robust End-to-end Speech Recognition](https://mlsanity.com/paper/2207.01732/) (2022)

## Abstract (arXiv)

Sequence-to-sequence models have shown success in end-to-end speech recognition. However these models have only used shallow acoustic encoder networks. In our work, we successively train very deep convolutional networks to add more expressive power and better generalization for end-to-end ASR models. We apply network-in-network principles, batch normalization, residual connections and convolutional LSTMs to build very deep recurrent and convolutional structures. Our models exploit the spectral structure in the feature space and add computational depth without overfitting issues. We experiment with the WSJ ASR task and achieve 10.5% word error rate without any dictionary or language using a 15 layer deep network.