mlsanityData to 14 Oct 2026

CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese

arXiv 2511.04139 ↗First posted 6 Nov 2025cs.CL cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

Automatic speech recognition (ASR) is critical for language accessibility, yet low-resource Cantonese remains challenging due to limited annotated data, six lexical tones, tone sandhi, and accent variation. Existing ASR models, such as Whisper, often suffer from high word error rates. Large audio-language models (LALMs), in contrast, can leverage broader contextual reasoning but still require explicit tonal and prosodic acoustic cues. We introduce CantoASR, a collaborative ASR-LALM error correction framework that integrates forced alignment for acoustic feature extraction, a LoRA-finetuned Whisper for improved tone discrimination, and an instruction-tuned Qwen-Audio for prosody-aware correction. Evaluations on spontaneous Cantonese data show substantial CER gains over Whisper-Large-V3. These findings suggest that integrating acoustic cues with LALM reasoning provides a scalable strategy for low-resource tonal and dialectal ASR.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

23 of 34 references.

References · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2024Qwen2-Audio Technical Reportcited by 560
  5. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  6. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  7. 2023Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Promptingcited by 72
  8. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  9. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 50
  10. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  11. 2023N-best T5: Robust ASR Error Correction using Multiple Input Hypotheses and Constrained Decoding Spacecited by 42
  12. 2024Towards audio language modeling -- an overviewcited by 40
11 more
  1. 2024ASR Error Correction using Large Language Modelscited by 24
  2. 2024Building a Taiwanese Mandarin Spoken Language Model: A First Attemptcited by 23
  3. 2021BART based semantic correction for Mandarin automatic speech recognition systemcited by 20
  4. 2022Hey ASR System! Why Aren't You More Inclusive? Automatic Speech Recognition Systems' Bias and Proposed Bias Mitigation Techniques. A Literature Reviewcited by 19
  5. 2022Automatic Speech Recognition Datasets in Cantonese: A Survey and New Datasetcited by 12
  6. 2024Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selectioncited by 10
  7. 2025HKCanto-Eval: A Benchmark for Evaluating Cantonese Language Understanding and Cultural Comprehension in LLMscited by 8
  8. 2025Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correctioncited by 7
  9. 2023Developing a Multilingual Dataset and Evaluation Metrics for Code-Switching: A Focus on Hong Kong's Polylingual Dynamicscited by 3
  10. 2024Full-text Error Correction for Chinese Speech Recognition with Large Language Modelcited by 3
  11. 2023Boosting Chinese ASR Error Correction with Dynamic Error Scaling Mechanismcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Modelscited by 8
  2. 2026VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speechcited by 3
  3. 2026Efficiently Adapting Spoken Language Models for the Singaporean Contextcited by 0