mlsanityData to 7 Oct 2026

Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata

arXiv 2309.09548 ↗First posted 18 Sep 2023eess.AS cs.LG cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Automated speech intelligibility assessment is pivotal for hearing aid (HA) development. In this paper, we present three novel methods to improve intelligibility prediction accuracy and introduce MBI-Net+, an enhanced version of MBI-Net, the top-performing system in the 1st Clarity Prediction Challenge. MBI-Net+ leverages Whisper's embeddings to create cross-domain acoustic features and includes metadata from speech signals by using a classifier that distinguishes different enhancement methods. Furthermore, MBI-Net+ integrates the hearing-aid speech perception index (HASPI) as a supplementary metric into the objective function to further boost prediction performance. Experimental results demonstrate that MBI-Net+ surpasses several intrusive baseline systems and MBI-Net on the Clarity Prediction Challenge 2023 dataset, validating the effectiveness of incorporating Whisper embeddings, speech metadata, and related complementary metrics to improve prediction performance for HA.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

16 of 35 references.

References · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  6. 2021Generalization Ability of MOS Prediction Networkscited by 69
  7. 2021MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Networkcited by 37
  8. 2021Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Featurescited by 32
  9. 2022ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applicationscited by 15
  10. 2022MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aidscited by 11
  11. 2022Fusion of Self-supervised Learned Models for MOS Predictioncited by 10
  12. 2022MTI-Net: A Multi-Target Speech Intelligibility Prediction Modelcited by 6
4 more
  1. 2022Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listenerscited by 4
  2. 2023The Effect of Spoken Language on Speech Enhancement using Self-Supervised Speech Representation Loss Functionscited by 3
  3. 2022Speech intelligibility of simulated hearing loss sounds and its prediction using the Gammachirp Envelope Similarity Index (GESI)cited by 1
  4. 2023Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Modelcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Transfer Learning from Whisper for Microscopic Intelligibility Predictioncited by 1
  2. 2025No Audiogram: Leveraging Existing Scores for Personalized Speech Intelligibility Predictioncited by 0
  3. 2025Non-Intrusive Binaural Speech Intelligibility Prediction Using Mamba for Hearing-Impaired Listenerscited by 0
  4. 2026Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Predictioncited by 0