# Letter-Based Speech Recognition with Gated ConvNets

arXiv: https://arxiv.org/abs/1712.09444 · HTML: https://mlsanity.com/paper/1712.09444/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2017.

## Built on by (most cited first, 6 of 6)

- [Towards Building ASR Systems for the Next Billion Users](https://mlsanity.com/paper/2111.03945/) (2021)
- [CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription](https://mlsanity.com/paper/2107.01091/) (2021)
- [End-to-End Automatic Speech Recognition model for the Sudanese Dialect](https://mlsanity.com/paper/2212.10826/) (2022)
- [Voice Quality and Pitch Features in Transformer-Based Speech Recognition](https://mlsanity.com/paper/2112.11391/) (2021)
- [Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting](https://mlsanity.com/paper/2210.09510/) (2022)
- [EmoAugNet: A Signal-Augmented Hybrid CNN-LSTM Framework for Speech Emotion Recognition](https://mlsanity.com/paper/2508.06321/) (2025)

## Abstract (arXiv)

In the recent literature, "end-to-end" speech systems often refer to letter-based acoustic models trained in a sequence-to-sequence manner, either via a recurrent model or via a structured output learning approach (such as CTC). In contrast to traditional phone (or senone)-based approaches, these "end-to-end'' approaches alleviate the need of word pronunciation modeling, and do not require a "forced alignment" step at training time. Phone-based approaches remain however state of the art on classical benchmarks. In this paper, we propose a letter-based speech recognition system, leveraging a ConvNet acoustic model. Key ingredients of the ConvNet are Gated Linear Units and high dropout. The ConvNet is trained to map audio sequences to their corresponding letter transcriptions, either via a classical CTC approach, or via a recent variant called ASG. Coupled with a simple decoder at inference time, our system matches the best existing letter-based systems on WSJ (in word error rate), and shows near state of the art performance on LibriSpeech.