mlsanityData to 7 Oct 2026

Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition

arXiv 2406.09873 ↗First posted 14 Jun 2024eess.AS cs.AI cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Disordered speech recognition profound implications for improving the quality of life for individuals afflicted with, for example, dysarthria. Dysarthric speech recognition encounters challenges including limited data, substantial dissimilarities between dysarthric and non-dysarthric speakers, and significant speaker variations stemming from the disorder. This paper introduces Perceiver-Prompt, a method for speaker adaptation that utilizes P-Tuning on the Whisper large-scale model. We first fine-tune Whisper using LoRA and then integrate a trainable Perceiver to generate fixed-length speaker prompts from variable-length inputs, to improve model recognition of Chinese dysarthric speech. Experimental results from our Chinese dysarthric speech dataset demonstrate consistent improvements in recognition performance with Perceiver-Prompt. Relative reduction up to 13.04% in CER is obtained over the fine-tuned Whisper.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

17 of 27 references.

References · 17

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  8. 2021GPT Understands, Toocited by 715
  9. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  10. 2019Personalizing ASR for Dysarthric and Accented Speech with Limited Datacited by 45
  11. 2016Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptationcited by 38
  12. 2022Speaker adaptation for Wav2vec2 based dysarthric ASRcited by 21
5 more
  1. 2022Recent Progress in the CUHK Dysarthric Speech Recognition Systemcited by 17
  2. 2023Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognitioncited by 14
  3. 2022Spectro-Temporal Deep Features for Disordered Speech Assessment and Recognitioncited by 13
  4. 2023Use of Speech Impairment Severity for Dysarthric Speech Recognitioncited by 8
  5. 2022On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognitioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Transcript-Prompted Whisper with Dictionary-Enhanced Decoding for Japanese Speech Annotationcited by 1
  2. 2025On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognitioncited by 0
  3. 2025EvolveCaptions: Empowering DHH Users Through Real-Time Collaborative Captioningcited by 0
  4. 2026Choosing a PEFT Variant for Per-Patient Dysarthric ASR: A Single-Speaker Case Study on Two ASR Basescited by 0