mlsanityData to 19 Oct 2026

A Perception-Based L2 Speech Intelligibility Indicator: Leveraging a Rater's Shadowing and Sequence-to-sequence Voice Conversion

arXiv 2505.24304 ↗First posted 30 May 2025eess.AS cs.SDcited by 0

On the map: Speech and audio language models → ASR evaluation, bias and atypical speech

Abstract (arXiv)

Evaluating L2 speech intelligibility is crucial for effective computer-assisted language learning (CALL). Conventional ASR-based methods often focus on native-likeness, which may fail to capture the actual intelligibility perceived by human listeners. In contrast, our work introduces a novel, perception based L2 speech intelligibility indicator that leverages a native rater's shadowing data within a sequence-to-sequence (seq2seq) voice conversion framework. By integrating an alignment mechanism and acoustic feature reconstruction, our approach simulates the auditory perception of native listeners, identifying segments in L2 speech that are likely to cause comprehension difficulties. Both objective and subjective evaluations indicate that our method aligns more closely with native judgments than traditional ASR-based metrics, offering a promising new direction for CALL systems in a global, multilingual contexts.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in ASR evaluation, bias and atypical speech, one of 11 directions in Speech and audio language models. Builds on 14 of its 30 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

14 of 30 references.

References · 14

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  4. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 552
  5. 2021One TTS Alignment To Rule Them Allcited by 63
  6. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 53
  7. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 49
  8. 2024VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Musiccited by 44
  9. 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 41
  10. 2024The VoiceMOS Challenge 2024: Beyond Speech Quality Predictioncited by 23
  11. 2021Non-autoregressive sequence-to-sequence voice conversioncited by 9
  12. 2023AAS-VC: On the Generalization Ability of Automatic Alignment Search based Non-autoregressive Sequence-to-sequence Voice Conversioncited by 4
2 more
  1. 2023Electrolaryngeal Speech Intelligibility Enhancement Through Robust Linguistic Encoderscited by 3
  2. 2024A Pilot Study of Applying Sequence-to-Sequence Voice Conversion to Evaluate the Intelligibility of L2 Speech Using a Native Speaker's Shadowingscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Simulating Native Speaker Shadowing for Nonnative Speech Assessment with Latent Speech Representations
  2. 2025Aligning ASR Evaluation with Human and LLM Judgments: Intelligibility Metrics Using Phonetic, Semantic, and NLI Approaches
  3. 2025Discrete Tokens Exhibit Interlanguage Speech Intelligibility Benefit: an Analytical Study Towards Accent-robust ASR Only with Native Speech Data
  4. 2026VocalGrad: Evaluating Acoustic Perception in Audio Language Models
  5. 2025HPSU: A Benchmark for Human-Level Perception in Real-World Spoken Speech Understanding
  6. 2023Investigating the Sensitivity of Automatic Speech Recognition Systems to Phonetic Variation in L2 Englishes
  7. 2025SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models
  8. 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokens