# Pay Attention to Hard Trials

arXiv: https://arxiv.org/abs/2209.04687 · HTML: https://mlsanity.com/paper/2209.04687/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 10 Sep 2022.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [CN-CELEB: a challenging Chinese speaker recognition dataset](https://mlsanity.com/paper/1911.01799/) (2019)
- [CN-Celeb: multi-genre speaker recognition](https://mlsanity.com/paper/2012.12468/) (2020)
- [VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge](https://mlsanity.com/paper/2012.06867/) (2020)
- [VoxSRC 2019: The first VoxCeleb Speaker Recognition Challenge](https://mlsanity.com/paper/1912.02522/) (2019)
- [Deep Speaker Feature Learning for Text-independent Speaker Verification](https://mlsanity.com/paper/1705.03670/) (2017)
- [C-P Map: A Novel Evaluation Toolkit for Speaker Verification](https://mlsanity.com/paper/2203.02942/) (2022)

## Built on by (most cited first, 1 of 1)

- [Speaker recognition with two-step multi-modal deep cleansing](https://mlsanity.com/paper/2210.15903/) (2022)

## Abstract (arXiv)

Performance of speaker recognition systems is evaluated on test trials. Although as crucial as rulers for tailors, trials have not been carefully treated so far, and most existing benchmarks compose trials by naive cross-pairing. In this paper, we argue that the cross-pairing approach produces overwhelming easy trials, which in turn leads to potential bias in system and technique comparison. To solve the problem, we advocate more attention to hard trials. We present an SVM-based approach to identifying hard trials and use it to construct new evaluation sets for VoxCeleb1 and SITW. With the new sets, we can re-evaluate the contribution of some recent technologies. The code and the identified hard trials will be published online at http://project.cslt.org.