# Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1701.02720 · HTML: https://mlsanity.com/paper/1701.02720/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2017.

## Built on by (most cited first, 24 of 24)

- [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022)
- [Zipformer: A faster and better encoder for automatic speech recognition](https://mlsanity.com/paper/2310.11230/) (2023)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Deep Model Intellectual Property Protection via Deep Watermarking](https://mlsanity.com/paper/2103.04980/) (2021)
- [Squeezeformer: An Efficient Transformer for Automatic Speech Recognition](https://mlsanity.com/paper/2206.00888/) (2022)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition](https://mlsanity.com/paper/2109.00627/) (2021)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization](https://mlsanity.com/paper/2304.14535/) (2023)
- [SECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inference](https://mlsanity.com/paper/2110.00478/) (2021)
- [Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generator](https://mlsanity.com/paper/2205.09058/) (2022)
- [Graph Convolutions Enrich the Self-Attention in Transformers!](https://mlsanity.com/paper/2312.04234/) (2023)
- [Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition](https://mlsanity.com/paper/2102.07259/) (2021)
- [Sequential Estimation of Gaussian Process-based Deep State-Space Models](https://mlsanity.com/paper/2301.12528/) (2023)
- [Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages](https://mlsanity.com/paper/2503.23542/) (2025)
- [Interpreting intermediate convolutional layers in unsupervised acoustic word classification](https://mlsanity.com/paper/2110.02375/) (2021)
- [End-to-End Spoken Language Understanding: Performance analyses of a voice command task in a low resource setting](https://mlsanity.com/paper/2207.08179/) (2022)
- [Quran Recitation Recognition using End-to-End Deep Learning](https://mlsanity.com/paper/2305.07034/) (2023)
- [Speaker-Independent Dysarthria Severity Classification using Self-Supervised Transformers and Multi-Task Learning](https://mlsanity.com/paper/2403.00854/) (2024)
- [Effects of Number of Filters of Convolutional Layers on Speech Recognition Model Accuracy](https://mlsanity.com/paper/2102.02326/) (2021)
- [Robustness of end-to-end Automatic Speech Recognition Models -- A Case Study using Mozilla DeepSpeech](https://mlsanity.com/paper/2105.09742/) (2021)
- [Time-Frequency Localization Using Deep Convolutional Maxout Neural Network in Persian Speech Recognition](https://mlsanity.com/paper/2108.03818/) (2021)
- [Automatic Speech Recognition for Greek Medical Dictation](https://mlsanity.com/paper/2509.23550/) (2025)
- [Quantization Range Estimation for Convolutional Neural Networks](https://mlsanity.com/paper/2510.04044/) (2025)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) are effective models for reducing spectral variations and modeling spectral correlations in acoustic features for automatic speech recognition (ASR). Hybrid speech recognition systems incorporating CNNs with Hidden Markov Models/Gaussian Mixture Models (HMMs/GMMs) have achieved the state-of-the-art in various benchmarks. Meanwhile, Connectionist Temporal Classification (CTC) with Recurrent Neural Networks (RNNs), which is proposed for labeling unsegmented sequences, makes it feasible to train an end-to-end speech recognition system instead of hybrid settings. However, RNNs are computationally expensive and sometimes difficult to train. In this paper, inspired by the advantages of both CNNs and the CTC approach, we propose an end-to-end speech framework for sequence labeling, by combining hierarchical CNNs with CTC directly without recurrent connections. By evaluating the approach on the TIMIT phoneme recognition task, we show that the proposed model is not only computationally efficient, but also competitive with the existing baseline systems. Moreover, we argue that CNNs have the capability to model temporal correlations with appropriate context information.