# Spectro-Temporal Deep Features for Disordered Speech Assessment and Recognition

arXiv: https://arxiv.org/abs/2201.05554 · HTML: https://mlsanity.com/paper/2201.05554/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 54 references that are in the map.
- First posted on arXiv 14 Jan 2022.

## Builds on (most cited first, 8 of 8)

- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Transformer-based Acoustic Modeling for Hybrid Speech Recognition](https://mlsanity.com/paper/1910.09799/) (2019)
- [Personalizing ASR for Dysarthric and Accented Speech with Limited Data](https://mlsanity.com/paper/1907.13511/) (2019)
- [Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation](https://mlsanity.com/paper/1601.02828/) (2016)
- [Improved End-to-End Dysarthric Speech Recognition via Meta-learning Based Model Re-initialization](https://mlsanity.com/paper/2011.01686/) (2020)
- [Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks](https://mlsanity.com/paper/2201.03943/) (2022)
- [Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition](https://mlsanity.com/paper/2012.04494/) (2020)

## Built on by (most cited first, 13 of 13)

- [Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2202.10290/) (2022)
- [Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2205.06445/) (2022)
- [Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection](https://mlsanity.com/paper/2206.13232/) (2022)
- [Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition](https://mlsanity.com/paper/2203.10274/) (2022)
- [Use of Speech Impairment Severity for Dysarthric Speech Recognition](https://mlsanity.com/paper/2305.10659/) (2023)
- [Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition](https://mlsanity.com/paper/2206.07327/) (2022)
- [Bottleneck Low-rank Transformers for Low-resource Spoken Language Understanding](https://mlsanity.com/paper/2206.14318/) (2022)
- [Automatic Severity Classification of Dysarthric speech by using Self-supervised Model with Multi-task Learning](https://mlsanity.com/paper/2210.15387/) (2022)
- [On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2203.14593/) (2022)
- [Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition](https://mlsanity.com/paper/2306.15265/) (2023)
- [Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition](https://mlsanity.com/paper/2406.09873/) (2024)
- [Debiased Automatic Speech Recognition for Dysarthric Speech via Sample Reweighting with Sample Affinity Test](https://mlsanity.com/paper/2305.13108/) (2023)
- [Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation](https://mlsanity.com/paper/2407.06310/) (2024)

## Abstract (arXiv)

Automatic recognition of disordered speech remains a highly challenging task to date. Sources of variability commonly found in normal speech including accent, age or gender, when further compounded with the underlying causes of speech impairment and varying severity levels, create large diversity among speakers. To this end, speaker adaptation techniques play a vital role in current speech recognition systems. Motivated by the spectro-temporal level differences between disordered and normal speech that systematically manifest in articulatory imprecision, decreased volume and clarity, slower speaking rates and increased dysfluencies, novel spectro-temporal subspace basis embedding deep features derived by SVD decomposition of speech spectrum are proposed to facilitate both accurate speech intelligibility assessment and auxiliary feature based speaker adaptation of state-of-the-art hybrid DNN and end-to-end disordered speech recognition systems. Experiments conducted on the UASpeech corpus suggest the proposed spectro-temporal deep feature adapted systems consistently outperformed baseline i-Vector adaptation by up to 2.63% absolute (8.6% relative) reduction in word error rate (WER) with or without data augmentation. Learning hidden unit contribution (LHUC) based speaker adaptation was further applied. The final speaker adapted system using the proposed spectral basis embedding features gave an overall WER of 25.6% on the UASpeech test set of 16 dysarthric speakers