# Deformable TDNN with adaptive receptive fields for speech recognition

arXiv: https://arxiv.org/abs/2104.14791 · HTML: https://mlsanity.com/paper/2104.14791/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Jasper: An End-to-End Convolutional Neural Acoustic Model](https://mlsanity.com/paper/1904.03288/) (2019)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [Espresso: A Fast End-to-end Neural Speech Recognition Toolkit](https://mlsanity.com/paper/1909.08723/) (2019)
- [Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture](https://mlsanity.com/paper/2001.08290/) (2020)
- [Fully Convolutional Speech Recognition](https://mlsanity.com/paper/1812.06864/) (2018)
- [Multistream CNN for Robust Acoustic Modeling](https://mlsanity.com/paper/2005.10470/) (2020)
- [CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency](https://mlsanity.com/paper/2005.13326/) (2020)
- [State-of-the-Art Speech Recognition Using Multi-Stream Self-Attention With Dilated 1D Convolutions](https://mlsanity.com/paper/1910.00716/) (2019)
- [Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients](https://mlsanity.com/paper/2011.05649/) (2020)
- [Feedforward Sequential Memory Neural Networks without Recurrent Feedback](https://mlsanity.com/paper/1510.02693/) (2015)

## Built on by (most cited first, 2 of 2)

- [Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers](https://mlsanity.com/paper/2107.03007/) (2021)
- [DEFORMER: Coupling Deformed Localized Patterns with Global Context for Robust End-to-end Speech Recognition](https://mlsanity.com/paper/2207.01732/) (2022)

## Abstract (arXiv)

Time Delay Neural Networks (TDNNs) are widely used in both DNN-HMM based hybrid speech recognition systems and recent end-to-end systems. Nevertheless, the receptive fields of TDNNs are limited and fixed, which is not desirable for tasks like speech recognition, where the temporal dynamics of speech are varied and affected by many factors. This paper proposes to use deformable TDNNs for adaptive temporal dynamics modeling in end-to-end speech recognition. Inspired by deformable ConvNets, deformable TDNNs augment the temporal sampling locations with additional offsets and learn the offsets automatically based on the ASR criterion, without additional supervision. Experiments show that deformable TDNNs obtain state-of-the-art results on WSJ benchmarks (1.42%/3.45% WER on WSJ eval92/dev93 respectively), outperforming standard TDNNs significantly. Furthermore, we propose the latency control mechanism for deformable TDNNs, which enables deformable TDNNs to do streaming ASR without accuracy degradation.