# Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representations

arXiv: https://arxiv.org/abs/2307.13423 · HTML: https://mlsanity.com/paper/2307.13423/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [Generalization Ability of MOS Prediction Networks](https://mlsanity.com/paper/2110.02635/) (2021)
- [ICASSP 2023 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2303.11510/) (2023)
- [Analysing Discrete Self Supervised Speech Representation for Spoken Language Modeling](https://mlsanity.com/paper/2301.00591/) (2023)
- [MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech](https://mlsanity.com/paper/2110.05866/) (2021)
- [MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids](https://mlsanity.com/paper/2204.03305/) (2022)
- [Perceive and predict: self-supervised speech representation based loss functions for speech enhancement](https://mlsanity.com/paper/2301.04388/) (2023)
- [Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applications](https://mlsanity.com/paper/2210.00259/) (2022)
- [MetricGAN+/-: Increasing Robustness of Noise Reduction on Unseen Data](https://mlsanity.com/paper/2203.12369/) (2022)
- [Exploring the influence of fine-tuning data on wav2vec 2.0 model for blind speech quality prediction](https://mlsanity.com/paper/2204.02135/) (2022)
- [Unsupervised Uncertainty Measures of Automatic Speech Recognition for Non-intrusive Speech Intelligibility Prediction](https://mlsanity.com/paper/2204.04288/) (2022)
- [Clarity: Machine Learning Challenges to Revolutionise Hearing Device Processing](https://mlsanity.com/paper/2006.11140/) (2020)
- [Non-Intrusive Binaural Speech Intelligibility Prediction from Discrete Latent Representations](https://mlsanity.com/paper/2111.12531/) (2021)

## Built on by (most cited first, 3 of 3)

- [Non-Intrusive Speech Intelligibility Prediction for Hearing-Impaired Users using Intermediate ASR Features and Human Memory Models](https://mlsanity.com/paper/2401.13611/) (2024)
- [Speech foundation models on intelligibility prediction for hearing-impaired listeners](https://mlsanity.com/paper/2401.14289/) (2024)
- [WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features](https://mlsanity.com/paper/2508.02210/) (2025)

## Abstract (arXiv)

Self-supervised speech representations (SSSRs) have been successfully applied to a number of speech-processing tasks, e.g. as feature extractor for speech quality (SQ) prediction, which is, in turn, relevant for assessment and training speech enhancement systems for users with normal or impaired hearing. However, exact knowledge of why and how quality-related information is encoded well in such representations remains poorly understood. In this work, techniques for non-intrusive prediction of SQ ratings are extended to the prediction of intelligibility for hearing-impaired users. It is found that self-supervised representations are useful as input features to non-intrusive prediction models, achieving competitive performance to more complex systems. A detailed analysis of the performance depending on Clarity Prediction Challenge 1 listeners and enhancement systems indicates that more data might be needed to allow generalisation to unknown systems and (hearing-impaired) individuals