# Speech foundation models in healthcare: Effect of layer selection on pathological speech feature prediction

arXiv: https://arxiv.org/abs/2402.01796 · HTML: https://mlsanity.com/paper/2402.01796/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [Exploration of A Self-Supervised Speech Model: A Study on Emotional Corpora](https://mlsanity.com/paper/2210.02595/) (2022)
- [What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysis](https://mlsanity.com/paper/2107.00439/) (2021)
- [Multi-class Detection of Pathological Speech with Latent Features: How does it perform on unseen data?](https://mlsanity.com/paper/2210.15336/) (2022)
- [Automatic Speaker Independent Dysarthric Speech Intelligibility Assessment System](https://mlsanity.com/paper/2103.06157/) (2021)
- [Transfer Learning for Personality Perception via Speech Emotion Recognition](https://mlsanity.com/paper/2305.16076/) (2023)
- [Detecting Speech Abnormalities with a Perceiver-based Sequence Classifier that Leverages a Universal Speech Model](https://mlsanity.com/paper/2310.13010/) (2023)

## Built on by (most cited first, 1 of 1)

- [Understanding Multilingual Medical ASR Adaptation Through Layer-Wise Analysis](https://mlsanity.com/paper/2608.18825/) (2026)

## Abstract (arXiv)

Accurately extracting clinical information from speech is critical to the diagnosis and treatment of many neurological conditions. As such, there is interest in leveraging AI for automatic, objective assessments of clinical speech to facilitate diagnosis and treatment of speech disorders. We explore transfer learning using foundation models, focusing on the impact of layer selection for the downstream task of predicting pathological speech features. We find that selecting an optimal layer can greatly improve performance (~15.8% increase in balanced accuracy per feature as compared to worst layer, ~13.6% increase as compared to final layer), though the best layer varies by predicted feature and does not always generalize well to unseen data. A learned weighted sum offers comparable performance to the average best layer in-distribution (only ~1.2% lower) and had strong generalization for out-of-distribution data (only 1.5% lower than the average best layer).