# Efficient Speech Quality Assessment using Self-supervised Framewise Embeddings

arXiv: https://arxiv.org/abs/2211.06646 · HTML: https://mlsanity.com/paper/2211.06646/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 12 Nov 2022.

## Builds on (most cited first, 23 of 23)

- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [The Efficiency Misnomer](https://mlsanity.com/paper/2110.12894/) (2021)
- [HEAR: Holistic Evaluation of Audio Representations](https://mlsanity.com/paper/2203.03022/) (2022)
- [CDPAM: Contrastive learning for perceptual audio similarity](https://mlsanity.com/paper/2102.05109/) (2021)
- [Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM](https://mlsanity.com/paper/1808.05344/) (2018)
- [SERAB: A multi-lingual benchmark for speech emotion recognition](https://mlsanity.com/paper/2110.03414/) (2021)
- [ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications](https://mlsanity.com/paper/2203.16032/) (2022)
- [SESQA: semi-supervised learning for speech quality assessment](https://mlsanity.com/paper/2010.00368/) (2020)
- [NORESQA: A Framework for Speech Quality Assessment using Non-Matching References](https://mlsanity.com/paper/2109.08125/) (2021)
- [Non-intrusive speech quality assessment using neural networks](https://mlsanity.com/paper/1903.06908/) (2019)
- [MetricNet: Towards Improved Modeling For Non-Intrusive Speech Quality Assessment](https://mlsanity.com/paper/2104.01227/) (2021)
- [BYOL-S: Learning Self-supervised Speech Representations by Bootstrapping](https://mlsanity.com/paper/2206.12038/) (2022)
- [DNN No-Reference PSTN Speech Quality Prediction](https://mlsanity.com/paper/2007.14598/) (2020)
- [Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applications](https://mlsanity.com/paper/2210.00259/) (2022)
- [Hybrid Handcrafted and Learnable Audio Representation for Analysis of Speech Under Cognitive and Physical Load](https://mlsanity.com/paper/2203.16637/) (2022)
- [A Universal Deep Room Acoustics Estimator](https://mlsanity.com/paper/2109.14436/) (2021)
- [Wav2Vec2.0 on the Edge: Performance Evaluation](https://mlsanity.com/paper/2202.05993/) (2022)
- [MOSRA: Joint Mean Opinion Score and Room Acoustics Speech Quality Assessment](https://mlsanity.com/paper/2204.01345/) (2022)

## Built on by (most cited first, 2 of 2)

- [Speaker Embeddings as Individuality Proxy for Voice Stress Detection](https://mlsanity.com/paper/2306.05915/) (2023)
- [A Feature Engineering Approach for Literary and Colloquial Tamil Speech Classification using 1D-CNN](https://mlsanity.com/paper/2409.14348/) (2024)

## Abstract (arXiv)

Automatic speech quality assessment is essential for audio researchers, developers, speech and language pathologists, and system quality engineers. The current state-of-the-art systems are based on framewise speech features (hand-engineered or learnable) combined with time dependency modeling. This paper proposes an efficient system with results comparable to the best performing model in the ConferencingSpeech 2022 challenge. Our proposed system is characterized by a smaller number of parameters (40-60x), fewer FLOPS (100x), lower memory consumption (10-15x), and lower latency (30x). Speech quality practitioners can therefore iterate much faster, deploy the system on resource-limited hardware, and, overall, the proposed system contributes to sustainable machine learning. The paper also concludes that framewise embeddings outperform utterance-level embeddings and that multi-task training with acoustic conditions modeling does not degrade speech quality prediction while providing better interpretation.