# Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features

arXiv: https://arxiv.org/abs/2111.02363 · HTML: https://mlsanity.com/paper/2111.02363/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 98 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [Generalization Ability of MOS Prediction Networks](https://mlsanity.com/paper/2110.02635/) (2021)
- [Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM](https://mlsanity.com/paper/1808.05344/) (2018)
- [MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network](https://mlsanity.com/paper/2103.00110/) (2021)
- [T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancement](https://mlsanity.com/paper/1910.06762/) (2019)
- [Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention](https://mlsanity.com/paper/2002.05873/) (2020)
- [Raw Waveform-based Speech Enhancement by Fully Convolutional Networks](https://mlsanity.com/paper/1703.02205/) (2017)
- [STOI-Net: A Deep Learning based Non-Intrusive Speech Intelligibility Assessment Model](https://mlsanity.com/paper/2011.04292/) (2020)
- [MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech](https://mlsanity.com/paper/2110.05866/) (2021)
- [Non-intrusive speech quality assessment using neural networks](https://mlsanity.com/paper/1903.06908/) (2019)
- [Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification](https://mlsanity.com/paper/2007.08267/) (2020)
- [A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals](https://mlsanity.com/paper/2007.15797/) (2020)
- [Speech Enhancement with Zero-Shot Model Selection](https://mlsanity.com/paper/2012.09359/) (2020)
- [Dynamic Noise Embedding: Noise Aware Training and Adaptation for Speech Enhancement](https://mlsanity.com/paper/2008.11920/) (2020)
- [SVSNet: An End-to-end Speaker Voice Similarity Assessment Model](https://mlsanity.com/paper/2107.09392/) (2021)

## Built on by (most cited first, 30 of 33)

- [Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition](https://mlsanity.com/paper/2307.10757/) (2023)
- [MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids](https://mlsanity.com/paper/2204.03305/) (2022)
- [Fusion of Self-supervised Learned Models for MOS Prediction](https://mlsanity.com/paper/2204.04855/) (2022)
- [SQuId: Measuring Speech Naturalness in Many Languages](https://mlsanity.com/paper/2210.06324/) (2022)
- [BSS-CFFMA: Cross-Domain Feature Fusion and Multi-Attention Speech Enhancement Network based on Self-Supervised Embedding](https://mlsanity.com/paper/2408.06851/) (2024)
- [DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores](https://mlsanity.com/paper/2204.03219/) (2022)
- [Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech](https://mlsanity.com/paper/2402.16321/) (2024)
- [MTI-Net: A Multi-Target Speech Intelligibility Prediction Model](https://mlsanity.com/paper/2204.03310/) (2022)
- [Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata](https://mlsanity.com/paper/2309.09548/) (2023)
- [Visual Realism Assessment for Face-swap Videos](https://mlsanity.com/paper/2302.00918/) (2023)
- [HAAQI-Net: A Non-intrusive Neural Music Audio Quality Assessment Model for Hearing Aids](https://mlsanity.com/paper/2401.01145/) (2024)
- [Bridging The Multi-Modality Gaps of Audio, Visual and Linguistic for Speech Enhancement](https://mlsanity.com/paper/2501.13375/) (2025)
- [Wideband Audio Waveform Evaluation Networks: Efficient, Accurate Estimation of Speech Qualities](https://mlsanity.com/paper/2206.13272/) (2022)
- [On the robustness of non-intrusive speech quality model by adversarial examples](https://mlsanity.com/paper/2211.06508/) (2022)
- [Evaluation of Speech Representations for MOS prediction](https://mlsanity.com/paper/2306.09979/) (2023)
- [Resource-Efficient Speech Quality Prediction through Quantization Aware Training and Binary Activation Maps](https://mlsanity.com/paper/2407.04578/) (2024)
- [NeuroAMP: A Novel End-to-end General Purpose Deep Neural Amplifier for Personalized Hearing Aids](https://mlsanity.com/paper/2502.10822/) (2025)
- [From Diet to Free Lunch: Estimating Auxiliary Signal Properties using Dynamic Pruning Masks in Speech Enhancement Networks](https://mlsanity.com/paper/2602.10666/) (2026)
- [Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Model](https://mlsanity.com/paper/2308.09262/) (2023)
- [CORN: Co-Trained Full- And No-Reference Speech Quality Assessment](https://mlsanity.com/paper/2310.09388/) (2023)
- [A Non-Intrusive Neural Quality Assessment Model for Surface Electromyography Signals](https://mlsanity.com/paper/2402.05482/) (2024)
- [Towards Robust Assessment of Pathological Voices via Combined Low-Level Descriptors and Foundation Model Representations](https://mlsanity.com/paper/2505.21356/) (2025)
- [Improving Perceptual Audio Aesthetic Assessment via Triplet Loss and Self-Supervised Embeddings](https://mlsanity.com/paper/2509.03292/) (2025)
- [MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module](https://mlsanity.com/paper/2301.07087/) (2023)
- [Speech Synthesis By Unrolling Diffusion Process using Neural Network Layers](https://mlsanity.com/paper/2309.09652/) (2023)
- [SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction](https://mlsanity.com/paper/2506.02082/) (2025)
- [MMMOS: Multi-domain Multi-axis Audio Quality Assessment](https://mlsanity.com/paper/2507.04094/) (2025)
- [MiSTR: Multi-Modal iEEG-to-Speech Synthesis with Transformer-Based Prosody Prediction and Neural Phase Reconstruction](https://mlsanity.com/paper/2508.03166/) (2025)
- [Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model](https://mlsanity.com/paper/2511.01868/) (2025)
- [Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction](https://mlsanity.com/paper/2602.15484/) (2026)

## Abstract (arXiv)

In this study, we propose a cross-domain multi-objective speech assessment model called MOSA-Net, which can estimate multiple speech assessment metrics simultaneously. Experimental results show that MOSA-Net can improve the linear correlation coefficient (LCC) by 0.026 (0.990 vs 0.964 in seen noise environments) and 0.012 (0.969 vs 0.957 in unseen noise environments) in perceptual evaluation of speech quality (PESQ) prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC by 0.021 (0.985 vs 0.964 in seen noise environments) and 0.047 (0.836 vs 0.789 in unseen noise environments) in short-time objective intelligibility (STOI) prediction, compared to STOI-Net (based on CRNN), an existing single-task model for STOI prediction. Moreover, MOSA-Net, originally trained to assess objective scores, can be used as a pre-trained model to be effectively adapted to an assessment model for predicting subjective quality and intelligibility scores with a limited amount of training data. Experimental results show that MOSA-Net can improve LCC by 0.018 (0.805 vs 0.787) in mean opinion score (MOS) prediction, compared to MOS-SSL, a strong single-task model for MOS prediction. In light of the confirmed prediction capability, we further adopt the latent representations of MOSA-Net to guide the speech enhancement (SE) process and derive a quality-intelligibility (QI)-aware SE (QIA-SE) approach accordingly. Experimental results show that QIA-SE provides superior enhancement performance compared with the baseline SE system in terms of objective evaluation metrics and qualitative evaluation test. For example, QIA-SE can improve PESQ by 0.301 (2.953 vs 2.652 in seen noise environments) and 0.18 (2.658 vs 2.478 in unseen noise environments) over a CNN-based baseline SE model.