# NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets

arXiv: https://arxiv.org/abs/2104.09494 · HTML: https://mlsanity.com/paper/2104.09494/

## Key facts

- Cited by 192 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric](https://mlsanity.com/paper/2004.09584/) (2020)
- [Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM](https://mlsanity.com/paper/1808.05344/) (2018)
- [An Open source Implementation of ITU-T Recommendation P.808 with Validation](https://mlsanity.com/paper/2005.08138/) (2020)
- [SESQA: semi-supervised learning for speech quality assessment](https://mlsanity.com/paper/2010.00368/) (2020)
- [Non-intrusive speech quality assessment using neural networks](https://mlsanity.com/paper/1903.06908/) (2019)
- [A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals](https://mlsanity.com/paper/2007.15797/) (2020)

## Built on by (most cited first, 30 of 192)

- [Recent Advances in Speech Language Models: A Survey](https://mlsanity.com/paper/2410.03751/) (2024)
- [VISTA: A Test-Time Self-Improving Video Generation Agent](https://mlsanity.com/paper/2510.15831/) (2025)
- [VABench: A Comprehensive Benchmark for Audio-Video Generation](https://mlsanity.com/paper/2512.09299/) (2025)
- [Audio Large Language Models Can Be Descriptive Speech Quality Evaluators](https://mlsanity.com/paper/2501.17202/) (2025)
- [T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation](https://mlsanity.com/paper/2512.21094/) (2025)
- [Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2412.06602/) (2024)
- [AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement](https://mlsanity.com/paper/2501.15417/) (2025)
- [AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation](https://mlsanity.com/paper/2507.12705/) (2025)
- [PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset](https://mlsanity.com/paper/2405.08838/) (2024)
- [Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation](https://mlsanity.com/paper/2409.16644/) (2024)
- [Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study](https://mlsanity.com/paper/2505.19598/) (2025)
- [Convert and Speak: Zero-shot Accent Conversion with Minimum Supervision](https://mlsanity.com/paper/2408.10096/) (2024)
- [EchoMind: An Interrelated Multi-level Benchmark for Evaluating Empathetic Speech Language Models](https://mlsanity.com/paper/2510.22758/) (2025)
- [Self-Supervised Learning for Speech Enhancement through Synthesis](https://mlsanity.com/paper/2211.02542/) (2022)
- [Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion](https://mlsanity.com/paper/2306.15875/) (2023)
- [Noise-robust Speech Separation with Fast Generative Correction](https://mlsanity.com/paper/2406.07461/) (2024)
- [SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline](https://mlsanity.com/paper/2505.19314/) (2025)
- [Perceive and predict: self-supervised speech representation based loss functions for speech enhancement](https://mlsanity.com/paper/2301.04388/) (2023)
- [Diffusion-based Generative Modeling with Discriminative Guidance for Streamable Speech Enhancement](https://mlsanity.com/paper/2406.13471/) (2024)
- [SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detection](https://mlsanity.com/paper/2407.18517/) (2024)
- [High-Resolution Speech Restoration with Latent Diffusion Model](https://mlsanity.com/paper/2409.11145/) (2024)
- [TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems](https://mlsanity.com/paper/2506.19441/) (2025)
- [InQSS: a speech intelligibility and quality assessment model using a multi-task learning network](https://mlsanity.com/paper/2111.02585/) (2021)
- [Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech](https://mlsanity.com/paper/2402.16321/) (2024)
- [DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Models](https://mlsanity.com/paper/2405.14632/) (2024)
- [Universal Speech Enhancement with Regression and Generative Mamba](https://mlsanity.com/paper/2505.21198/) (2025)
- [Multi-Metric Preference Alignment for Generative Speech Restoration](https://mlsanity.com/paper/2508.17229/) (2025)
- [Towards General-Purpose Text-Instruction-Guided Voice Conversion](https://mlsanity.com/paper/2309.14324/) (2023)
- [Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Model](https://mlsanity.com/paper/2402.10642/) (2024)
- [Learn from Real: Reality Defender's Submission to ASVspoof5 Challenge](https://mlsanity.com/paper/2410.07379/) (2024)

## Abstract (arXiv)

In this paper, we present an update to the NISQA speech quality prediction model that is focused on distortions that occur in communication networks. In contrast to the previous version, the model is trained end-to-end and the time-dependency modelling and time-pooling is achieved through a Self-Attention mechanism. Besides overall speech quality, the model also predicts the four speech quality dimensions Noisiness, Coloration, Discontinuity, and Loudness, and in this way gives more insight into the cause of a quality degradation. Furthermore, new datasets with over 13,000 speech files were created for training and validation of the model. The model was finally tested on a new, live-talking test dataset that contains recordings of real telephone calls. Overall, NISQA was trained and evaluated on 81 datasets from different sources and showed to provide reliable predictions also for unknown speech samples. The code, model weights, and datasets are open-sourced.