# The PyTorch-Kaldi Speech Recognition Toolkit

arXiv: https://arxiv.org/abs/1811.07453 · HTML: https://mlsanity.com/paper/1811.07453/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2018.

## Built on by (most cited first, 30 of 40)

- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [WaveGuard: Understanding and Mitigating Audio Adversarial Examples](https://mlsanity.com/paper/2103.03344/) (2021)
- [DialogueNeRF: Towards Realistic Avatar Face-to-Face Conversation Video Generation](https://mlsanity.com/paper/2203.07931/) (2022)
- [Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning](https://mlsanity.com/paper/2109.01905/) (2021)
- [Defending against Adversarial Audio via Diffusion Model](https://mlsanity.com/paper/2303.01507/) (2023)
- [Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-Temporal Sparsity](https://mlsanity.com/paper/2108.02297/) (2021)
- [BembaSpeech: A Speech Recognition Corpus for the Bemba Language](https://mlsanity.com/paper/2102.04889/) (2021)
- [Semantic-WER: A Unified Metric for the Evaluation of ASR Transcript for End Usability](https://mlsanity.com/paper/2106.02016/) (2021)
- [Synthesizing Dysarthric Speech Using Multi-talker TTS for Dysarthric Speech Recognition](https://mlsanity.com/paper/2201.11571/) (2022)
- [Bangla Natural Language Processing: A Comprehensive Analysis of Classical, Machine Learning, and Deep Learning Based Methods](https://mlsanity.com/paper/2105.14875/) (2021)
- [Sequential End-to-End Intent and Slot Label Classification and Localization](https://mlsanity.com/paper/2106.04660/) (2021)
- [Accurate synthesis of Dysarthric Speech for ASR data augmentation](https://mlsanity.com/paper/2308.08438/) (2023)
- [Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2104.00120/) (2021)
- [LRWR: Large-Scale Benchmark for Lip Reading in Russian language](https://mlsanity.com/paper/2109.06692/) (2021)
- [Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gate](https://mlsanity.com/paper/2310.14982/) (2023)
- [Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows](https://mlsanity.com/paper/2102.07284/) (2021)
- [Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability](https://mlsanity.com/paper/2107.00730/) (2021)
- [SVSNet: An End-to-end Speaker Voice Similarity Assessment Model](https://mlsanity.com/paper/2107.09392/) (2021)
- [Seq2seq for Automatic Paraphasia Detection in Aphasic Speech](https://mlsanity.com/paper/2312.10518/) (2023)
- [Transferable Adversarial Attacks against ASR](https://mlsanity.com/paper/2411.09220/) (2024)
- [Towards Natural and Controllable Cross-Lingual Voice Conversion Based on Neural TTS Model and Phonetic Posteriorgram](https://mlsanity.com/paper/2102.01991/) (2021)
- [Memory Planning for Deep Neural Networks](https://mlsanity.com/paper/2203.00448/) (2022)
- [Multi-style Training for South African Call Centre Audio](https://mlsanity.com/paper/2202.07219/) (2022)
- [ASR Benchmarking: Need for a More Representative Conversational Dataset](https://mlsanity.com/paper/2409.12042/) (2024)
- [Inclusive Speaker Verification with Adaptive thresholding](https://mlsanity.com/paper/2111.05501/) (2021)
- [Efficient acoustic feature transformation in mismatched environments using a Guided-GAN](https://mlsanity.com/paper/2210.00721/) (2022)
- [Improved Contextual Recognition In Automatic Speech Recognition Systems By Semantic Lattice Rescoring](https://mlsanity.com/paper/2310.09680/) (2023)
- [A 71.2-μW Speech Recognition Accelerator with Recurrent Spiking Neural Network](https://mlsanity.com/paper/2503.21337/) (2025)

## Abstract (arXiv)

The availability of open-source software is playing a remarkable role in the popularization of speech recognition and deep learning. Kaldi, for instance, is nowadays an established framework used to develop state-of-the-art speech recognizers. PyTorch is used to build neural networks with the Python language and has recently spawn tremendous interest within the machine learning community thanks to its simplicity and flexibility. The PyTorch-Kaldi project aims to bridge the gap between these popular toolkits, trying to inherit the efficiency of Kaldi and the flexibility of PyTorch. PyTorch-Kaldi is not only a simple interface between these software, but it embeds several useful features for developing modern speech recognizers. For instance, the code is specifically designed to naturally plug-in user-defined acoustic models. As an alternative, users can exploit several pre-implemented neural networks that can be customized using intuitive configuration files. PyTorch-Kaldi supports multiple feature and label streams as well as combinations of neural networks, enabling the use of complex neural architectures. The toolkit is publicly-released along with a rich documentation and is designed to properly work locally or on HPC clusters. Experiments, that are conducted on several datasets and tasks, show that PyTorch-Kaldi can effectively be used to develop modern state-of-the-art speech recognizers.