# SHEET: A Multi-purpose Open-source Speech Human Evaluation Estimation Toolkit

arXiv: https://arxiv.org/abs/2505.15061 · HTML: https://mlsanity.com/paper/2505.15061/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2025.

## Built on by (most cited first, 4 of 4)

- [AudioMosaic: Contrastive Masked Audio Representation Learning](https://mlsanity.com/paper/2605.14231/) (2026)
- [Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations](https://mlsanity.com/paper/2606.19951/) (2026)
- [PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors](https://mlsanity.com/paper/2606.20137/) (2026)
- [Listen to the Features: Voice Anonymization Driven by Content Embedding Matching over Signal Reconstruction](https://mlsanity.com/paper/2607.09767/) (2026)

## Abstract (arXiv)

We introduce SHEET, a multi-purpose open-source toolkit designed to accelerate subjective speech quality assessment (SSQA) research. SHEET stands for the Speech Human Evaluation Estimation Toolkit, which focuses on data-driven deep neural network-based models trained to predict human-labeled quality scores of speech samples. SHEET provides comprehensive training and evaluation scripts, multi-dataset and multi-model support, as well as pre-trained models accessible via Torch Hub and HuggingFace Spaces. To demonstrate its capabilities, we re-evaluated SSL-MOS, a speech self-supervised learning (SSL)-based SSQA model widely used in recent scientific papers, on an extensive list of speech SSL models. Experiments were conducted on two representative SSQA datasets named BVCC and NISQA, and we identified the optimal speech SSL model, whose performance surpassed the original SSL-MOS implementation and was comparable to state-of-the-art methods.