# Non-Intrusive Speech Intelligibility Prediction for Hearing-Impaired Users using Intermediate ASR Features and Human Memory Models

arXiv: https://arxiv.org/abs/2401.13611 · HTML: https://mlsanity.com/paper/2401.13611/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids](https://mlsanity.com/paper/2204.03305/) (2022)
- [Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applications](https://mlsanity.com/paper/2210.00259/) (2022)
- [Unsupervised Uncertainty Measures of Automatic Speech Recognition for Non-intrusive Speech Intelligibility Prediction](https://mlsanity.com/paper/2204.04288/) (2022)
- [Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representations](https://mlsanity.com/paper/2307.13423/) (2023)

## Built on by (most cited first, 9 of 9)

- [Unveiling the Best Practices for Applying Speech Foundation Models to Speech Intelligibility Prediction for Hearing-Impaired People](https://mlsanity.com/paper/2505.08215/) (2025)
- [WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features](https://mlsanity.com/paper/2508.02210/) (2025)
- [Transfer Learning from Whisper for Microscopic Intelligibility Prediction](https://mlsanity.com/paper/2404.01737/) (2024)
- [Transcription-Free Fine-Tuning of Speech Separation Models for Noisy and Reverberant Multi-Speaker Automatic Speech Recognition](https://mlsanity.com/paper/2406.08914/) (2024)
- [Improving Perceptual Audio Aesthetic Assessment via Triplet Loss and Self-Supervised Embeddings](https://mlsanity.com/paper/2509.03292/) (2025)
- [No Audiogram: Leveraging Existing Scores for Personalized Speech Intelligibility Prediction](https://mlsanity.com/paper/2506.02039/) (2025)
- [Non-Intrusive Binaural Speech Intelligibility Prediction Using Mamba for Hearing-Impaired Listeners](https://mlsanity.com/paper/2507.05729/) (2025)
- [Whilter: A Whisper-based Data Filter for "In-the-Wild" Speech Corpora Using Utterance-level Multi-Task Classification](https://mlsanity.com/paper/2507.21642/) (2025)
- [Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction](https://mlsanity.com/paper/2602.15484/) (2026)

## Abstract (arXiv)

Neural networks have been successfully used for non-intrusive speech intelligibility prediction. Recently, the use of feature representations sourced from intermediate layers of pre-trained self-supervised and weakly-supervised models has been found to be particularly useful for this task. This work combines the use of Whisper ASR decoder layer representations as neural network input features with an exemplar-based, psychologically motivated model of human memory to predict human intelligibility ratings for hearing-aid users. Substantial performance improvement over an established intrusive HASPI baseline system is found, including on enhancement systems and listeners unseen in the training data, with a root mean squared error of 25.3 compared with the baseline of 28.7.