# Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions

arXiv: https://arxiv.org/abs/1904.02619 · HTML: https://mlsanity.com/paper/1904.02619/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2019.

## Built on by (most cited first, 27 of 27)

- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation](https://mlsanity.com/paper/2412.02725/) (2024)
- [emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface Electromyography](https://mlsanity.com/paper/2410.20081/) (2024)
- [Acoustic Data-Driven Subword Modeling for End-to-End Speech Recognition](https://mlsanity.com/paper/2104.09106/) (2021)
- [Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder](https://mlsanity.com/paper/2107.00189/) (2021)
- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)
- [Supervised Attention in Sequence-to-Sequence Models for Speech Recognition](https://mlsanity.com/paper/2204.12308/) (2022)
- [RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition](https://mlsanity.com/paper/2305.17782/) (2023)
- [LLM-Driven Multimodal Opinion Expression Identification](https://mlsanity.com/paper/2406.18088/) (2024)
- [Scaling and Distilling Transformer Models for sEMG](https://mlsanity.com/paper/2507.22094/) (2025)
- [emg2speech: Synthesizing speech from electromyography using self-supervised speech models](https://mlsanity.com/paper/2510.23969/) (2025)
- [EgoEMG: A Multimodal Egocentric Dataset with Bilateral EMG and Vision for Hand Pose Estimation](https://mlsanity.com/paper/2605.05712/) (2026)
- [Deconvolution-and-convolution Networks](https://mlsanity.com/paper/2103.11887/) (2021)
- [Compressing 1D Time-Channel Separable Convolutions using Sparse Random Ternary Matrices](https://mlsanity.com/paper/2103.17142/) (2021)
- [SpeeChain: A Speech Toolkit for Large-Scale Machine Speech Chain](https://mlsanity.com/paper/2301.02966/) (2023)
- [Let SSMs be ConvNets: State-space Modeling with Optimal Tensor Contractions](https://mlsanity.com/paper/2501.13230/) (2025)
- [Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Training](https://mlsanity.com/paper/2210.16892/) (2022)
- [Self-consistent context aware conformer transducer for speech recognition](https://mlsanity.com/paper/2402.06592/) (2024)
- [SplashNet: Split-and-Share Encoders for Accurate and Efficient Typing with Surface Electromyography](https://mlsanity.com/paper/2506.12356/) (2025)
- [Voice Quality and Pitch Features in Transformer-Based Speech Recognition](https://mlsanity.com/paper/2112.11391/) (2021)
- [Mixture-of-Rookies: Saving DNN Computations by Predicting ReLU Outputs](https://mlsanity.com/paper/2202.04990/) (2022)
- [A Comparison of Transformer, Convolutional, and Recurrent Neural Networks on Phoneme Recognition](https://mlsanity.com/paper/2210.00367/) (2022)
- [Typing Reinvented: Towards Hands-Free Input via sEMG](https://mlsanity.com/paper/2511.18213/) (2025)
- [From Muscle to Text with MyoText: sEMG to Text via Finger Classification and Transformer-Based Decoding](https://mlsanity.com/paper/2601.03098/) (2026)
- [Myovox: Reading Speech from the Muscles of the Face](https://mlsanity.com/paper/2609.17548/) (2026)
- [EFormer: Temporally Aligned Local Correction for Continuous sEMG-Based Hand Pose Tracking](https://mlsanity.com/paper/2609.38932/) (2026)

## Abstract (arXiv)

We propose a fully convolutional sequence-to-sequence encoder architecture with a simple and efficient decoder. Our model improves WER on LibriSpeech while being an order of magnitude more efficient than a strong RNN baseline. Key to our approach is a time-depth separable convolution block which dramatically reduces the number of parameters in the model while keeping the receptive field large. We also give a stable and efficient beam search inference procedure which allows us to effectively integrate a language model. Coupled with a convolutional language model, our time-depth separable convolution architecture improves by more than 22% relative WER over the best previously reported sequence-to-sequence results on the noisy LibriSpeech test set.