# Effects of Layer Freezing on Transferring a Speech Recognition System to Under-resourced Languages

arXiv: https://arxiv.org/abs/2102.04097 · HTML: https://mlsanity.com/paper/2102.04097/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems](https://mlsanity.com/paper/1709.04482/) (2017)
- [Transfer Learning for Speech Recognition on a Budget](https://mlsanity.com/paper/1706.00290/) (2017)
- [Cantonese Automatic Speech Recognition Using Transfer Learning from Mandarin](https://mlsanity.com/paper/1911.09271/) (2019)

## Built on by (most cited first, 3 of 3)

- [ADI-20: Arabic Dialect Identification dataset and models](https://mlsanity.com/paper/2511.10070/) (2025)
- [Accelerating Transfer Learning with Near-Data Computation on Cloud Object Stores](https://mlsanity.com/paper/2210.08650/) (2022)
- [Evaluating the Effectiveness of Pre-trained Language Models in Predicting the Helpfulness of Online Product Reviews](https://mlsanity.com/paper/2302.10199/) (2023)

## Abstract (arXiv)

In this paper, we investigate the effect of layer freezing on the effectiveness of model transfer in the area of automatic speech recognition. We experiment with Mozilla's DeepSpeech architecture on German and Swiss German speech datasets and compare the results of either training from scratch vs. transferring a pre-trained model. We compare different layer freezing schemes and find that even freezing only one layer already significantly improves results.