# Whispering in Amharic: Fine-tuning Whisper for Low-resource Language

arXiv: https://arxiv.org/abs/2503.18485 · HTML: https://mlsanity.com/paper/2503.18485/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [The Revolution of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2402.12451/) (2024)
- [Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach](https://mlsanity.com/paper/2010.07835/) (2020)
- [Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey](https://mlsanity.com/paper/2403.01255/) (2024)
- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)
- [Natural Language Processing in Ethiopian Languages: Current State, Challenges, and Opportunities](https://mlsanity.com/paper/2303.14406/) (2023)
- [Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges](https://mlsanity.com/paper/2402.01917/) (2024)
- [Improving Whisper's Recognition Performance for Under-Represented Language Kazakh Leveraging Unpaired Speech and Text](https://mlsanity.com/paper/2408.05554/) (2024)

## Built on by (most cited first, 5 of 5)

- [A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data](https://mlsanity.com/paper/2506.11130/) (2025)
- [Structured Sparsity and Weight-adaptive Pruning for Memory and Compute efficient Whisper models](https://mlsanity.com/paper/2510.12666/) (2025)
- [Fine-tuning Whisper for Pashto ASR: strategies and scale](https://mlsanity.com/paper/2604.06507/) (2026)
- [Which Languages Transfer Best to Warlpiri? A Similarity-Based Study for Low-Resource ASR](https://mlsanity.com/paper/2607.10256/) (2026)
- [myMediWhisper: Construction of Burmese Medical Speech Corpus and Whisper Fine-Tuning for Clinical Dialogue ASR](https://mlsanity.com/paper/2608.11036/) (2026)

## Abstract (arXiv)

This work explores fine-tuning OpenAI's Whisper automatic speech recognition (ASR) model for Amharic, a low-resource language, to improve transcription accuracy. While the foundational Whisper model struggles with Amharic due to limited representation in its training data, we fine-tune it using datasets like Mozilla Common Voice, FLEURS, and the BDU-speech dataset. The best-performing model, Whispersmall-am, significantly improves when finetuned on a mix of existing FLEURS data and new, unseen Amharic datasets. Training solely on new data leads to poor performance, but combining it with FLEURS data reinforces the model, enabling better specialization in Amharic. We also demonstrate that normalizing Amharic homophones significantly enhances Word Error Rate (WER) and Bilingual Evaluation Understudy (BLEU) scores. This study underscores the importance of fine-tuning strategies and dataset composition for improving ASR in low-resource languages, providing insights for future Amharic speech recognition research.