# A New Benchmark of Aphasia Speech Recognition and Detection Based on E-Branchformer and Multi-task Learning

arXiv: https://arxiv.org/abs/2305.13331 · HTML: https://mlsanity.com/paper/2305.13331/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 22 of 22)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge](https://mlsanity.com/paper/2004.06833/) (2020)
- [Intermediate Loss Regularization for CTC-based Speech Recognition](https://mlsanity.com/paper/2102.03216/) (2021)
- [E-Branchformer: Branchformer with Enhanced merging for speech recognition](https://mlsanity.com/paper/2210.00077/) (2022)
- [Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding](https://mlsanity.com/paper/2207.02971/) (2022)
- [Multilingual Speech Recognition With A Single End-To-End Model](https://mlsanity.com/paper/1711.01694/) (2017)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [Improving Massively Multilingual ASR With Auxiliary CTC Objectives](https://mlsanity.com/paper/2302.12829/) (2023)
- [Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages](https://mlsanity.com/paper/1806.05059/) (2018)
- [Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection](https://mlsanity.com/paper/2206.13232/) (2022)
- [Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition](https://mlsanity.com/paper/2206.07327/) (2022)
- [End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representation](https://mlsanity.com/paper/2210.10742/) (2022)
- [Cross-Language Aphasia Detection using Optimal Transport Domain Adaptation](https://mlsanity.com/paper/1912.04370/) (2019)
- [Aphasic Speech Recognition using a Mixture of Speech Intelligibility Experts](https://mlsanity.com/paper/2008.10788/) (2020)
- [Zero-Shot Cross-lingual Aphasia Detection using Automatic Speech Recognition](https://mlsanity.com/paper/2204.00448/) (2022)
- [An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia](https://mlsanity.com/paper/1904.00361/) (2019)

## Built on by (most cited first, 4 of 4)

- [Seq2seq for Automatic Paraphasia Detection in Aphasic Speech](https://mlsanity.com/paper/2312.10518/) (2023)
- [Low-resource speech recognition and dialect identification of Irish in a multi-task framework](https://mlsanity.com/paper/2405.01293/) (2024)
- [Automatic Speech Recognition Advancements for Indigenous Languages of the Americas](https://mlsanity.com/paper/2404.08368/) (2024)
- [AS-ASR: A Lightweight Framework for Aphasia-Specific Automatic Speech Recognition](https://mlsanity.com/paper/2506.06566/) (2025)

## Abstract (arXiv)

Aphasia is a language disorder that affects the speaking ability of millions of patients. This paper presents a new benchmark for Aphasia speech recognition and detection tasks using state-of-the-art speech recognition techniques with the AphsiaBank dataset. Specifically, we introduce two multi-task learning methods based on the CTC/Attention architecture to perform both tasks simultaneously. Our system achieves state-of-the-art speaker-level detection accuracy (97.3%), and a relative WER reduction of 11% for moderate Aphasia patients. In addition, we demonstrate the generalizability of our approach by applying it to another disordered speech database, the DementiaBank Pitt corpus. We will make our all-in-one recipes and pre-trained model publicly available to facilitate reproducibility. Our standardized data preprocessing pipeline and open-source recipes enable researchers to compare results directly, promoting progress in disordered speech processing.