# Speaker recognition with two-step multi-modal deep cleansing

arXiv: https://arxiv.org/abs/2210.15903 · HTML: https://mlsanity.com/paper/2210.15903/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2022.

## Built on by (most cited first, 6 of 6)

- [Audio-Visual Speaker Verification via Joint Cross-Attention](https://mlsanity.com/paper/2309.16569/) (2023)
- [Audio-Visual Person Verification based on Recursive Fusion of Joint Cross-Attention](https://mlsanity.com/paper/2403.04654/) (2024)
- [Dynamic Cross Attention for Audio-Visual Person Verification](https://mlsanity.com/paper/2403.04661/) (2024)
- [Shared Multi-modal Embedding Space for Face-Voice Association](https://mlsanity.com/paper/2512.04814/) (2025)
- [Getting More for Less: Using Weak Labels and AV-Mixup for Robust Audio-Visual Speaker Verification](https://mlsanity.com/paper/2309.07115/) (2023)
- [SSAVSV: Towards Unified Model for Self-Supervised Audio-Visual Speaker Verification](https://mlsanity.com/paper/2506.17694/) (2025)

## Abstract (arXiv)

Neural network-based speaker recognition has achieved significant improvement in recent years. A robust speaker representation learns meaningful knowledge from both hard and easy samples in the training set to achieve good performance. However, noisy samples (i.e., with wrong labels) in the training set induce confusion and cause the network to learn the incorrect representation. In this paper, we propose a two-step audio-visual deep cleansing framework to eliminate the effect of noisy labels in speaker representation learning. This framework contains a coarse-grained cleansing step to search for the peculiar samples, followed by a fine-grained cleansing step to filter out the noisy labels. Our study starts from an efficient audio-visual speaker recognition system, which achieves a close to perfect equal-error-rate (EER) of 0.01%, 0.07% and 0.13% on the Vox-O, E and H test sets. With the proposed multi-modal cleansing mechanism, four different speaker recognition networks achieve an average improvement of 5.9%. Code has been made available at: textcolormagentaurlhttps://github.com/TaoRuijie/AVCleanse.