# Careful Whisper -- leveraging advances in automatic speech recognition for robust and interpretable aphasia subtype classification

arXiv: https://arxiv.org/abs/2308.01327 · HTML: https://mlsanity.com/paper/2308.01327/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates](https://mlsanity.com/paper/1911.03167/) (2019)
- [CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition](https://mlsanity.com/paper/2007.09127/) (2020)
- [CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation](https://mlsanity.com/paper/2204.00862/) (2022)
- [Cross-Language Aphasia Detection using Optimal Transport Domain Adaptation](https://mlsanity.com/paper/1912.04370/) (2019)
- [Zero-Shot Cross-lingual Aphasia Detection using Automatic Speech Recognition](https://mlsanity.com/paper/2204.00448/) (2022)

## Built on by (most cited first, 5 of 5)

- [CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions](https://mlsanity.com/paper/2408.16589/) (2024)
- [Devising a Set of Compact and Explainable Spoken Language Feature for Screening Alzheimer's Disease](https://mlsanity.com/paper/2411.18922/) (2024)
- [Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing](https://mlsanity.com/paper/2607.18934/) (2026)
- [Intended Target Identification for Anomia Patients with Gradient-based Selective Augmentation](https://mlsanity.com/paper/2506.14203/) (2025)
- [Bridging neuroscience and AI: adaptive, culturally sensitive technologies transforming aphasia rehabilitation](https://mlsanity.com/paper/2603.22357/) (2026)

## Abstract (arXiv)

This paper presents a fully automated approach for identifying speech anomalies from voice recordings to aid in the assessment of speech impairments. By combining Connectionist Temporal Classification (CTC) and encoder-decoder-based automatic speech recognition models, we generate rich acoustic and clean transcripts. We then apply several natural language processing methods to extract features from these transcripts to produce prototypes of healthy speech. Basic distance measures from these prototypes serve as input features for standard machine learning classifiers, yielding human-level accuracy for the distinction between recordings of people with aphasia and a healthy control group. Furthermore, the most frequently occurring aphasia types can be distinguished with 90% accuracy. The pipeline is directly applicable to other diseases and languages, showing promise for robustly extracting diagnostic speech biomarkers.