# Transfer Learning for Speech Recognition on a Budget

arXiv: https://arxiv.org/abs/1706.00290 · HTML: https://mlsanity.com/paper/1706.00290/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2017.

## Built on by (most cited first, 25 of 25)

- [UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data](https://mlsanity.com/paper/2101.07597/) (2021)
- [ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets](https://mlsanity.com/paper/2406.08641/) (2024)
- [OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation](https://mlsanity.com/paper/2107.00249/) (2021)
- [Using Radio Archives for Low-Resource Speech Recognition: Towards an Intelligent Virtual Assistant for Illiterate Users](https://mlsanity.com/paper/2104.13083/) (2021)
- [Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition](https://mlsanity.com/paper/2109.09161/) (2021)
- [Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification](https://mlsanity.com/paper/2108.02598/) (2021)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Improving speech recognition models with small samples for air traffic control systems](https://mlsanity.com/paper/2102.08015/) (2021)
- [Baby Physical Safety Monitoring in Smart Home Using Action Recognition System](https://mlsanity.com/paper/2210.12527/) (2022)
- [Effects of Layer Freezing on Transferring a Speech Recognition System to Under-resourced Languages](https://mlsanity.com/paper/2102.04097/) (2021)
- [Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding](https://mlsanity.com/paper/2202.05209/) (2022)
- [Improving Whisper's Recognition Performance for Under-Represented Language Kazakh Leveraging Unpaired Speech and Text](https://mlsanity.com/paper/2408.05554/) (2024)
- [Semi-supervised transfer learning for language expansion of end-to-end speech recognition models to low-resource languages](https://mlsanity.com/paper/2111.10047/) (2021)
- [Towards scalable efficient on-device ASR with transfer learning](https://mlsanity.com/paper/2407.16664/) (2024)
- [Streaming Bilingual End-to-End ASR model using Attention over Multiple Softmax](https://mlsanity.com/paper/2401.11645/) (2024)
- [Automatic Speech Recognition Advancements for Indigenous Languages of the Americas](https://mlsanity.com/paper/2404.08368/) (2024)
- [Enhancing Indonesian Automatic Speech Recognition: Evaluating Multilingual Models with Diverse Speech Variabilities](https://mlsanity.com/paper/2410.08828/) (2024)
- [Coarse-To-Fine And Cross-Lingual ASR Transfer](https://mlsanity.com/paper/2109.00916/) (2021)
- [WavFT: Acoustic model finetuning with labelled and unlabelled data](https://mlsanity.com/paper/2204.00348/) (2022)
- [Deploying self-supervised learning in the wild for hybrid automatic speech recognition](https://mlsanity.com/paper/2205.08598/) (2022)
- [Adaptive Activation Network For Low Resource Multilingual Speech Recognition](https://mlsanity.com/paper/2205.14326/) (2022)
- [Deep transfer learning for detecting Covid-19, Pneumonia and Tuberculosis using CXR images -- A Review](https://mlsanity.com/paper/2303.16754/) (2023)
- [Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition](https://mlsanity.com/paper/2310.13015/) (2023)
- [A Comprehensive Study of the Current State-of-the-Art in Nepali Automatic Speech Recognition Systems](https://mlsanity.com/paper/2402.03050/) (2024)
- [Building English ASR model with regional language support](https://mlsanity.com/paper/2503.07522/) (2025)

## Abstract (arXiv)

End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wav2Letter convolutional neural network originally trained for English ASR to the German language. We show that this technique allows faster training on consumer-grade resources while requiring less training data in order to achieve the same accuracy, thereby lowering the cost of training ASR models in other languages. Model introspection revealed that small adaptations to the network's weights were sufficient for good performance, especially for inner layers.