# AutoSpeech: Neural Architecture Search for Speaker Recognition

arXiv: https://arxiv.org/abs/2005.03215 · HTML: https://mlsanity.com/paper/2005.03215/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 17 of 17)

- [VoxCeleb Enrichment for Age and Gender Recognition](https://mlsanity.com/paper/2109.13510/) (2021)
- [Generalization Properties of NAS under Activation and Skip Connection Search](https://mlsanity.com/paper/2209.07238/) (2022)
- [AS2T: Arbitrary Source-To-Target Adversarial Attack on Speaker Recognition Systems](https://mlsanity.com/paper/2206.03351/) (2022)
- [Partially-Connected Differentiable Architecture Search for Deepfake and Spoofing Detection](https://mlsanity.com/paper/2104.03123/) (2021)
- [QFA2SR: Query-Free Adversarial Transfer Attacks to Speaker Recognition Systems](https://mlsanity.com/paper/2305.14097/) (2023)
- [Towards Dog Bark Decoding: Leveraging Human Speech Processing for Automated Bark Classification](https://mlsanity.com/paper/2404.18739/) (2024)
- [Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search](https://mlsanity.com/paper/2304.05405/) (2023)
- [SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song Covers](https://mlsanity.com/paper/2401.17133/) (2024)
- [Graph-based Label Propagation for Semi-Supervised Speaker Identification](https://mlsanity.com/paper/2106.08207/) (2021)
- [Demographic Attributes Prediction from Speech Using WavLM Embeddings](https://mlsanity.com/paper/2502.12007/) (2025)
- [EfficientTDNN: Efficient Architecture Search for Speaker Recognition](https://mlsanity.com/paper/2103.13581/) (2021)
- [Evaluating the Practicality of Learned Image Compression](https://mlsanity.com/paper/2207.14524/) (2022)
- [SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification](https://mlsanity.com/paper/2109.08839/) (2021)
- [Defend Data Poisoning Attacks on Voice Authentication](https://mlsanity.com/paper/2209.04547/) (2022)
- [NAS-VAD: Neural Architecture Search for Voice Activity Detection](https://mlsanity.com/paper/2201.09032/) (2022)
- [Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines](https://mlsanity.com/paper/2310.07940/) (2023)
- [Straight Through Gumbel Softmax Estimator based Bimodal Neural Architecture Search for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2406.13384/) (2024)

## Abstract (arXiv)

Speaker recognition systems based on Convolutional Neural Networks (CNNs) are often built with off-the-shelf backbones such as VGG-Net or ResNet. However, these backbones were originally proposed for image classification, and therefore may not be naturally fit for speaker recognition. Due to the prohibitive complexity of manually exploring the design space, we propose the first neural architecture search approach approach for the speaker recognition tasks, named as AutoSpeech. Our algorithm first identifies the optimal operation combination in a neural cell and then derives a CNN model by stacking the neural cell for multiple times. The final speaker recognition model can be obtained by training the derived CNN model through the standard scheme. To evaluate the proposed approach, we conduct experiments on both speaker identification and speaker verification tasks using the VoxCeleb1 dataset. Results demonstrate that the derived CNN architectures from the proposed approach significantly outperform current speaker recognition systems based on VGG-M, ResNet-18, and ResNet-34 back-bones, while enjoying lower model complexity.