# Seq2seq for Automatic Paraphasia Detection in Aphasic Speech

arXiv: https://arxiv.org/abs/2312.10518 · HTML: https://mlsanity.com/paper/2312.10518/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 53 references that are in the map.
- First posted on arXiv 16 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Automatic Detection of Fake News](https://mlsanity.com/paper/1708.07104/) (2017)
- [Exploring Neural Transducers for End-to-End Speech Recognition](https://mlsanity.com/paper/1707.07413/) (2017)
- [The PyTorch-Kaldi Speech Recognition Toolkit](https://mlsanity.com/paper/1811.07453/) (2018)
- [A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Tasks](https://mlsanity.com/paper/2305.11073/) (2023)
- [Aphasic Speech Recognition using a Mixture of Speech Intelligibility Experts](https://mlsanity.com/paper/2008.10788/) (2020)
- [A New Benchmark of Aphasia Speech Recognition and Detection Based on E-Branchformer and Multi-task Learning](https://mlsanity.com/paper/2305.13331/) (2023)

## Built on by (most cited first, 4 of 4)

- [Accent-Invariant Automatic Speech Recognition via Saliency-Driven Spectrogram Masking](https://mlsanity.com/paper/2510.09528/) (2025)
- [Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect](https://mlsanity.com/paper/2505.21809/) (2025)
- [Beyond Binary: Multiclass Paraphasia Detection with Generative Pretrained Transformers and End-to-End Models](https://mlsanity.com/paper/2407.11345/) (2024)
- [Generating Completions for Broca's Aphasic Sentences Using Large Language Models](https://mlsanity.com/paper/2412.17669/) (2024)

## Abstract (arXiv)

Paraphasias are speech errors that are often characteristic of aphasia and they represent an important signal in assessing disease severity and subtype. Traditionally, clinicians manually identify paraphasias by transcribing and analyzing speech-language samples, which can be a time-consuming and burdensome process. Identifying paraphasias automatically can greatly help clinicians with the transcription process and ultimately facilitate more efficient and consistent aphasia assessment. Previous research has demonstrated the feasibility of automatic paraphasia detection by training an automatic speech recognition (ASR) model to extract transcripts and then training a separate paraphasia detection model on a set of hand-engineered features. In this paper, we propose a novel, sequence-to-sequence (seq2seq) model that is trained end-to-end (E2E) to perform both ASR and paraphasia detection tasks. We show that the proposed model outperforms the previous state-of-the-art approach for both word-level and utterance-level paraphasia detection tasks and provide additional follow-up evaluations to further understand the proposed model behavior.