# A large-scale study of performance and equity of commercial remote identity verification technologies across demographics

arXiv: https://arxiv.org/abs/2409.12318 · HTML: https://mlsanity.com/paper/2409.12318/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Demographic Bias in Biometrics: A Survey on an Emerging Challenge](https://mlsanity.com/paper/2003.02488/) (2020)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [Mitigating Face Recognition Bias via Group Adaptive Classifier](https://mlsanity.com/paper/2006.07576/) (2020)
- [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019)
- [Fairness in Biometrics: a figure of merit to assess biometric verification systems](https://mlsanity.com/paper/2011.02395/) (2020)
- [On the Robustness of Face Recognition Algorithms Against Attacks and Bias](https://mlsanity.com/paper/2002.02942/) (2020)
- [Measuring Hidden Bias within Face Recognition via Racial Phenotypes](https://mlsanity.com/paper/2110.09839/) (2021)
- [Information-Theoretic Bias Assessment Of Learned Representations Of Pretrained Face Recognition](https://mlsanity.com/paper/2111.04673/) (2021)
- [Statistical Methods for Assessing Differences in False Non-Match Rates Across Demographic Groups](https://mlsanity.com/paper/2208.10948/) (2022)

## Built on by (most cited first, 4 of 4)

- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Selfie-Capture Dynamics as an Auxiliary Signal Against Deepfakes and Injection Attacks for Mobile Identity Verification](https://mlsanity.com/paper/2605.00218/) (2026)
- [Robustness of Presentation Attack Detection in Remote Identity Validation Scenarios](https://mlsanity.com/paper/2602.00109/) (2026)
- [IDSPACE: A Novel Document Generator for Reliable Evaluation of Digital Identity Verification Systems [Extended Technical Report]](https://mlsanity.com/paper/2609.03052/) (2026)

## Abstract (arXiv)

As more types of transactions move online, there is an increasing need to verify someone's identity remotely. Remote identity verification (RIdV) technologies have emerged to fill this need. RIdV solutions typically use a smart device to validate an identity document like a driver's license by comparing a face selfie to the face photo on the document. Recent research has been focused on ensuring that biometric systems work fairly across demographic groups. This study assesses five commercial RIdV solutions for equity across age, gender, race/ethnicity, and skin tone across 3,991 test subjects. This paper employs statistical methods to discern whether the RIdV result across demographic groups is statistically distinguishable. Two of the RIdV solutions were equitable across all demographics, while two RIdV solutions had at least one demographic that was inequitable. For example, the results for one technology had a false negative rate of 10.5% +/- 4.5% and its performance for each demographic category was within the error bounds, and, hence, were equitable. The other technologies saw either poor overall performance or inequitable performance. For one of these, participants of the race Black/African American (B/AA) as well as those with darker skin tones (Monk scale 7/8/9/10) experienced higher false rejections. Finally, one technology demonstrated more favorable but inequitable performance for the Asian American and Pacific Islander (AAPI) demographic. This study confirms that it is necessary to evaluate products across demographic groups to fully understand the performance of remote identity verification technologies.