# Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet

arXiv: https://arxiv.org/abs/2406.17825 · HTML: https://mlsanity.com/paper/2406.17825/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Achieving Human Parity in Conversational Speech Recognition](https://mlsanity.com/paper/1610.05256/) (2016)
- [PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition](https://mlsanity.com/paper/2106.05933/) (2021)
- [Light Gated Recurrent Units for Speech Recognition](https://mlsanity.com/paper/1803.10225/) (2018)
- [An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.04590/) (2021)
- [Very Deep Convolutional Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1610.03022/) (2016)
- [Relative Positional Encoding for Speech Recognition and Direct Translation](https://mlsanity.com/paper/2005.09940/) (2020)
- [Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1410.4281/) (2014)

## Built on by (most cited first, 3 of 3)

- [Artificial intelligence-enabled detection and assessment of Parkinson's disease using multimodal data: A survey](https://mlsanity.com/paper/2502.10703/) (2025)
- [Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration](https://mlsanity.com/paper/2602.21647/) (2026)
- [Nw=ach=a Mun=a: A Devanagari Speech Corpus and Proximal Transfer Benchmark for Nepal Bhasha ASR](https://mlsanity.com/paper/2603.07554/) (2026)

## Abstract (arXiv)

This paper presents an end-to-end deep learning model for Automatic Speech Recognition (ASR) that transcribes Nepali speech to text. The model was trained and tested on the OpenSLR (audio, text) dataset. The majority of the audio dataset have silent gaps at both ends which are clipped during dataset preprocessing for a more uniform mapping of audio frames and their corresponding texts. Mel Frequency Cepstral Coefficients (MFCCs) are used as audio features to feed into the model. The model having Bidirectional LSTM paired with ResNet and one-dimensional CNN produces the best results for this dataset out of all the models (neural networks with variations of LSTM, GRU, CNN, and ResNet) that have been trained so far. This novel model uses Connectionist Temporal Classification (CTC) function for loss calculation during training and CTC beam search decoding for predicting characters as the most likely sequence of Nepali text. On the test dataset, the character error rate (CER) of 17.06 percent has been achieved. The source code is available at: https://github.com/manishdhakal/ASR-Nepali-using-CNN-BiLSTM-ResNet.