mlsanityData to 14 Oct 2026

T-SANDHI: Tone Sandhi-aware Adaptive Network with Decoupled Hybrid Injection for Low-resource Taiwanese Hokkien Speech Recognition

arXiv 2609.18194 ↗First posted 16 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In Taiwanese Hokkien automatic speech recognition (ASR), prior studies often treat tone sandhi as a major challenge under the assumption that models fail to process implicit phonological variations. However, our experiments on Taiwanese Hokkien reveal that speech foundation models actually handle tone sandhi variations effectively, and the real performance bottleneck stems from a localized confusion between these variations and retained citation tones. To address this, we propose T-SANDHI to explicitly decouple surface acoustics from underlying lexical intent on top of a frozen Whisper backbone. Using a lexicon-guided multi-task learning structure driven by text-derived pseudo labels, our lightweight hybrid injection module integrates independent citation and sandhi phonetic streams via dynamic gating. Extensive evaluation on the TAT-MOE corpus and two blind test sets demonstrates that this explicit disentanglement effectively resolves tonal mapping confusion, outperforming baselines with strict parameter efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

19 of 40 references.

References · 19

  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  2. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  3. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 357
  4. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  5. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  6. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  7. 2021AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenariocited by 60
  8. 2024LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASRcited by 36
  9. 2023Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentationcited by 31
  10. 2022Speech-to-Speech Translation For A Real-world Unwritten Languagecited by 25
  11. 2024Encoding of lexical tone in self-supervised models of spoken languagecited by 23
  12. 2023Sparsely Shared LoRA on Whisper for Child Speech Recognitioncited by 18
7 more
  1. 2020Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Convertercited by 16
  2. 2023Evaluating Self-supervised Speech Models on a Taiwanese Hokkien Corpuscited by 7
  3. 2025A Practitioner's Guide to Building ASR Models for Low-Resource Languages: A Case Study on Scottish Gaeliccited by 5
  4. 2025Speech Recognition for Automatically Assessing Afrikaans and isiXhosa Preschool Oral Narrativescited by 4
  5. 2025CLiFT-ASR: A Cross-Lingual Fine-Tuning Framework for Low-Resource Taiwanese Hokkien Speech Recognitioncited by 2
  6. 2024Enhancing CTC-based speech recognition with diverse modeling unitscited by 1
  7. 2025Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SITA: Learning Speaker-Invariant and Tone-Aware Speech Representations for Low-Resource Tonal Languages
  2. 2025BreezyVoice: Adapting TTS for Taiwanese Mandarin with Enhanced Polyphone Disambiguation -- Challenges and Insights
  3. 2024ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams
  4. 2026Token Merging for Multilingual Speech Recognition: A Systematic Study Across Model Scale and Fine-Tuning
  5. 2026Bolbosh: Script-Aware Flow Matching for Kashmiri Text-to-Speech
  6. 2025Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models
  7. 2026FlowEdit: Associative Memory for Lifelong Pronunciation Adaptation in Flow-Matching TTS
  8. 2025A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data