# LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech

arXiv: https://arxiv.org/abs/2110.09103 · HTML: https://mlsanity.com/paper/2110.09103/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020)
- [Generalization Ability of MOS Prediction Networks](https://mlsanity.com/paper/2110.02635/) (2021)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit](https://mlsanity.com/paper/1910.10909/) (2019)
- [MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network](https://mlsanity.com/paper/2103.00110/) (2021)
- [Deep Learning Based Assessment of Synthetic Speech Naturalness](https://mlsanity.com/paper/2104.11673/) (2021)
- [How do Voices from Past Speech Synthesis Challenges Compare Today?](https://mlsanity.com/paper/2105.02373/) (2021)
- [Utilizing Self-supervised Representations for MOS Prediction](https://mlsanity.com/paper/2104.03017/) (2021)
- [MetricNet: Towards Improved Modeling For Non-Intrusive Speech Quality Assessment](https://mlsanity.com/paper/2104.01227/) (2021)

## Built on by (most cited first, 24 of 24)

- [DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs](https://mlsanity.com/paper/2201.11972/) (2022)
- [SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis](https://mlsanity.com/paper/2204.03040/) (2022)
- [Translatotron 3: Speech to Speech Translation with Monolingual Data](https://mlsanity.com/paper/2305.17547/) (2023)
- [Fusion of Self-supervised Learned Models for MOS Prediction](https://mlsanity.com/paper/2204.04855/) (2022)
- [SQuId: Measuring Speech Naturalness in Many Languages](https://mlsanity.com/paper/2210.06324/) (2022)
- [DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores](https://mlsanity.com/paper/2204.03219/) (2022)
- [MOSPC: MOS Prediction Based on Pairwise Comparison](https://mlsanity.com/paper/2306.10493/) (2023)
- [Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages](https://mlsanity.com/paper/2305.19396/) (2023)
- [Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Features](https://mlsanity.com/paper/2211.00342/) (2022)
- [Automatic Speech Recognition System-Independent Word Error Rate Estimation](https://mlsanity.com/paper/2404.16743/) (2024)
- [APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech](https://mlsanity.com/paper/2504.20447/) (2025)
- [ORCA: Open-ended Response Correctness Assessment for Audio Question Answering](https://mlsanity.com/paper/2512.09066/) (2025)
- [Using Rater and System Metadata to Explain Variance in the VoiceMOS Challenge 2022 Dataset](https://mlsanity.com/paper/2209.06358/) (2022)
- [Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Model](https://mlsanity.com/paper/2308.09262/) (2023)
- [Improving Perceptual Audio Aesthetic Assessment via Triplet Loss and Self-Supervised Embeddings](https://mlsanity.com/paper/2509.03292/) (2025)
- [Comparison of Speech Representations for the MOS Prediction System](https://mlsanity.com/paper/2206.13817/) (2022)
- [MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module](https://mlsanity.com/paper/2301.07087/) (2023)
- [Personalized Audio Quality Preference Prediction](https://mlsanity.com/paper/2302.08130/) (2023)
- [Speech Synthesis By Unrolling Diffusion Process using Neural Network Layers](https://mlsanity.com/paper/2309.09652/) (2023)
- [SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction](https://mlsanity.com/paper/2506.02082/) (2025)
- [MMMOS: Multi-domain Multi-axis Audio Quality Assessment](https://mlsanity.com/paper/2507.04094/) (2025)
- [EMA-SAM: Exponential Moving-average for SAM-based PTMC Segmentation](https://mlsanity.com/paper/2510.18213/) (2025)
- [DeRA-MOS: Optimizing Text-to-Music Evaluation via Decoupled Listwise Ranking and Modality Alignment](https://mlsanity.com/paper/2606.10010/) (2026)
- [DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization](https://mlsanity.com/paper/2608.21176/) (2026)

## Abstract (arXiv)

An effective approach to automatically predict the subjective rating for synthetic speech is to train on a listening test dataset with human-annotated scores. Although each speech sample in the dataset is rated by several listeners, most previous works only used the mean score as the training target. In this work, we present LDNet, a unified framework for mean opinion score (MOS) prediction that predicts the listener-wise perceived quality given the input speech and the listener identity. We reflect recent advances in LD modeling, including design choices of the model architecture, and propose two inference methods that provide more stable results and efficient computation. We conduct systematic experiments on the voice conversion challenge (VCC) 2018 benchmark and a newly collected large-scale MOS dataset, providing an in-depth analysis of the proposed framework. Results show that the mean listener inference method is a better way to utilize the mean scores, whose effectiveness is more obvious when having more ratings per sample.