mlsanityData to 19 Oct 2026

Pay Attention to Hard Trials

arXiv 2209.04687 ↗First posted 10 Sep 2022cs.SD eess.AScited by 1

On the map: Speech and audio language models → Speaker recognition and diarization

Abstract (arXiv)

Performance of speaker recognition systems is evaluated on test trials. Although as crucial as rulers for tailors, trials have not been carefully treated so far, and most existing benchmarks compose trials by naive cross-pairing. In this paper, we argue that the cross-pairing approach produces overwhelming easy trials, which in turn leads to potential bias in system and technique comparison. To solve the problem, we advocate more attention to hard trials. We present an SVM-based approach to identifying hard trials and use it to construct new evaluation sets for VoxCeleb1 and SITW. With the new sets, we can re-evaluate the contribution of some recent technologies. The code and the identified hard trials will be published online at http://project.cslt.org.

Key facts. Cited by 1 paper in the map (data through 19 Oct 2026). Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models. Builds on 12 of its 43 references that are in the map. First posted on arXiv 10 Sep 2022.

Builds on

12 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022C-P Map: A Novel Evaluation Toolkit for Speaker Verification6 Mar

Other references · 11

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,456
  2. 2018VoxCeleb2: Deep Speaker Recognitioncited by 1,166
  3. 2017VoxCeleb: a large-scale speaker identification datasetcited by 985
  4. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 980
  5. 2018Additive Margin Softmax for Face Verificationcited by 326
  6. 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 207
  7. 2019CN-CELEB: a challenging Chinese speaker recognition datasetcited by 123
  8. 2020CN-Celeb: multi-genre speaker recognitioncited by 66
  9. 2020VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challengecited by 46
  10. 2019VoxSRC 2019: The first VoxCeleb Speaker Recognition Challengecited by 25
  11. 2017Deep Speaker Feature Learning for Text-independent Speaker Verificationcited by 11

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Speaker recognition with two-step multi-modal deep cleansingcited by 9

Similar, not cited

Close in wording, no citation either way

  1. 2024VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark
  2. 2024Asymmetric and trial-dependent modeling: the contribution of LIA to SdSV Challenge Task 2
  3. 2024The VoxCeleb Speaker Recognition Challenge: A Retrospective
  4. 2022Design Guidelines for Inclusive Speaker Verification Evaluation Datasets
  5. 2025SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions
  6. 2022Bias in Automated Speaker Recognition
  7. 2026InterBias-SV: Compound Conditions in Speaker Verification
  8. 2022Large-scale learning of generalised representations for speaker recognition