# Speaker Recognition Based on Deep Learning: An Overview

arXiv: https://arxiv.org/abs/2012.00931 · HTML: https://mlsanity.com/paper/2012.00931/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2020.

## Built on by (most cited first, 30 of 59)

- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [A Survey of Mamba](https://mlsanity.com/paper/2408.01129/) (2024)
- [Bias in Automated Speaker Recognition](https://mlsanity.com/paper/2201.09486/) (2022)
- [Deep neural network enabled corrective source term approach to hybrid analysis and modeling](https://mlsanity.com/paper/2105.11521/) (2021)
- [Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings](https://mlsanity.com/paper/2203.16685/) (2022)
- [SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems](https://mlsanity.com/paper/2309.07983/) (2023)
- [A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker One](https://mlsanity.com/paper/2302.09908/) (2023)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders](https://mlsanity.com/paper/2305.10734/) (2023)
- [Towards single integrated spoofing-aware speaker verification embeddings](https://mlsanity.com/paper/2305.19051/) (2023)
- [StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Models](https://mlsanity.com/paper/2311.16509/) (2023)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [Dictionary Attacks on Speaker Verification](https://mlsanity.com/paper/2204.11304/) (2022)
- [Invisible Backdoor Attacks Using Data Poisoning in the Frequency Domain](https://mlsanity.com/paper/2207.04209/) (2022)
- [End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge](https://mlsanity.com/paper/2207.10817/) (2022)
- [SPGISpeech 2.0: Transcribed multi-speaker financial audio for speaker-tagged transcription](https://mlsanity.com/paper/2508.05554/) (2025)
- [HiddenSpeaker: Generate Imperceptible Unlearnable Audios for Speaker Verification System](https://mlsanity.com/paper/2405.15655/) (2024)
- [The Unreliability of Acoustic Systems in Alzheimer's Speech Datasets with Heterogeneous Recording Conditions](https://mlsanity.com/paper/2409.12170/) (2024)
- [Real Additive Margin Softmax for Speaker Verification](https://mlsanity.com/paper/2110.09116/) (2021)
- [Design Guidelines for Inclusive Speaker Verification Evaluation Datasets](https://mlsanity.com/paper/2204.02281/) (2022)
- [Towards an Interpretable Representation of Speaker Identity via Perceptual Voice Qualities](https://mlsanity.com/paper/2310.02497/) (2023)
- [Generalizing Speaker Verification for Spoof Awareness in the Embedding Space](https://mlsanity.com/paper/2401.11156/) (2024)
- [Self-Tuning Spectral Clustering for Speaker Diarization](https://mlsanity.com/paper/2410.00023/) (2024)
- [CBW: Towards Dataset Ownership Verification for Speaker Verification via Clustering-based Backdoor Watermarking](https://mlsanity.com/paper/2503.05794/) (2025)
- [U-vectors: Generating clusterable speaker embedding from unlabeled data](https://mlsanity.com/paper/2102.03868/) (2021)
- [Optimized Power Normalized Cepstral Coefficients towards Robust Deep Speaker Verification](https://mlsanity.com/paper/2109.12058/) (2021)
- [Spoofing-Aware Speaker Verification with Unsupervised Domain Adaptation](https://mlsanity.com/paper/2203.10992/) (2022)
- [Automatic Evaluation of Speaker Similarity](https://mlsanity.com/paper/2207.00344/) (2022)
- [Defend Data Poisoning Attacks on Voice Authentication](https://mlsanity.com/paper/2209.04547/) (2022)
- [LMD: A Learnable Mask Network to Detect Adversarial Examples for Speaker Verification](https://mlsanity.com/paper/2211.00825/) (2022)

## Abstract (arXiv)

Speaker recognition is a task of identifying persons from their voices. Recently, deep learning has dramatically revolutionized speaker recognition. However, there is lack of comprehensive reviews on the exciting progress. In this paper, we review several major subtasks of speaker recognition, including speaker verification, identification, diarization, and robust speaker recognition, with a focus on deep-learning-based methods. Because the major advantage of deep learning over conventional methods is its representation ability, which is able to produce highly abstract embedding features from utterances, we first pay close attention to deep-learning-based speaker feature extraction, including the inputs, network structures, temporal pooling strategies, and objective functions respectively, which are the fundamental components of many speaker recognition subtasks. Then, we make an overview of speaker diarization, with an emphasis of recent supervised, end-to-end, and online diarization. Finally, we survey robust speaker recognition from the perspectives of domain adaptation and speech enhancement, which are two major approaches of dealing with domain mismatch and noise problems. Popular and recently released corpora are listed at the end of the paper.