# Automatic Evaluation of Speaker Similarity

arXiv: https://arxiv.org/abs/2207.00344 · HTML: https://mlsanity.com/paper/2207.00344/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Parallel WaveNet: Fast High-Fidelity Speech Synthesis](https://mlsanity.com/paper/1711.10433/) (2017)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [Hierarchical Generative Modeling for Controllable Speech Synthesis](https://mlsanity.com/paper/1810.07217/) (2018)
- [Deep Voice 2: Multi-Speaker Neural Text-to-Speech](https://mlsanity.com/paper/1705.08947/) (2017)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [Speaker Recognition Based on Deep Learning: An Overview](https://mlsanity.com/paper/2012.00931/) (2020)
- [Deep Speaker: an End-to-End Neural Speaker Embedding System](https://mlsanity.com/paper/1705.02304/) (2017)
- [Low-resource expressive text-to-speech using data augmentation](https://mlsanity.com/paper/2011.05707/) (2020)
- [Universal Neural Vocoding with Parallel WaveNet](https://mlsanity.com/paper/2102.01106/) (2021)
- [A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences](https://mlsanity.com/paper/2001.04460/) (2020)
- [AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech](https://mlsanity.com/paper/1611.09207/) (2016)
- [CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speech](https://mlsanity.com/paper/2004.14617/) (2020)
- [Centroid-based deep metric learning for speaker recognition](https://mlsanity.com/paper/1902.02375/) (2019)
- [Attention-Based Models for Text-Dependent Speaker Verification](https://mlsanity.com/paper/1710.10470/) (2017)
- [Non-intrusive speech quality assessment using neural networks](https://mlsanity.com/paper/1903.06908/) (2019)
- [Improving multi-speaker TTS prosody variance with a residual encoder and normalizing flows](https://mlsanity.com/paper/2106.05762/) (2021)
- [SVSNet: An End-to-end Speaker Voice Similarity Assessment Model](https://mlsanity.com/paper/2107.09392/) (2021)

## Built on by (most cited first, 2 of 2)

- [Analyzing and Improving Speaker Similarity Assessment for Speech Synthesis](https://mlsanity.com/paper/2507.02176/) (2025)
- [Acoustic and perceptual differences between standard and accented speech and their voice clones](https://mlsanity.com/paper/2604.01562/) (2026)

## Abstract (arXiv)

We introduce a new automatic evaluation method for speaker similarity assessment, that is consistent with human perceptual scores. Modern neural text-to-speech models require a vast amount of clean training data, which is why many solutions switch from single speaker models to solutions trained on examples from many different speakers. Multi-speaker models bring new possibilities, such as a faster creation of new voices, but also a new problem - speaker leakage, where the speaker identity of a synthesized example might not match those of the target speaker. Currently, the only way to discover this issue is through costly perceptual evaluations. In this work, we propose an automatic method for assessment of speaker similarity. For that purpose, we extend the recent work on speaker verification systems and evaluate how different metrics and speaker embeddings models reflect Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) scores. Our experiments show that we can train a model to predict speaker similarity MUSHRA scores from speaker embeddings with 0.96 accuracy and significant correlation up to 0.78 Pearson score at the utterance level.