# Beyond Error-vs-Discard Characteristic: Toward Stable and Reliable Evaluation for Face Image Quality Assessment

arXiv: https://arxiv.org/abs/2607.22752 · HTML: https://mlsanity.com/paper/2607.22752/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 21 of 21)

- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/1708.08197/) (2017)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness](https://mlsanity.com/paper/2003.09373/) (2020)
- [Face Image Quality Assessment: A Literature Survey](https://mlsanity.com/paper/2009.01103/) (2020)
- [CR-FIQA: Face Image Quality Assessment by Learning Sample Relative Classifiability](https://mlsanity.com/paper/2112.06592/) (2021)
- [SDD-FIQA: Unsupervised Face Image Quality Assessment with Similarity Distribution Distance](https://mlsanity.com/paper/2103.05977/) (2021)
- [FaceQnet: Quality Assessment for Face Recognition based on Deep Learning](https://mlsanity.com/paper/1904.01740/) (2019)
- [Fairness in Biometrics: a figure of merit to assess biometric verification systems](https://mlsanity.com/paper/2011.02395/) (2020)
- [SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation](https://mlsanity.com/paper/2308.11509/) (2023)
- [Cross-Quality LFW: A Database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/2108.10290/) (2021)
- [Face Quality Estimation and Its Correlation to Demographic and Non-Demographic Bias in Face Recognition](https://mlsanity.com/paper/2004.01019/) (2020)
- [QMagFace: Simple and Accurate Quality-Aware Face Recognition](https://mlsanity.com/paper/2111.13475/) (2021)
- [DifFIQA: Face Image Quality Assessment Using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2305.05768/) (2023)
- [FaceQAN: Face Image Quality Assessment Through Adversarial Noise Exploration](https://mlsanity.com/paper/2212.02127/) (2022)
- [Considerations on the Evaluation of Biometric Quality Assessment Algorithms](https://mlsanity.com/paper/2303.13294/) (2023)
- [GraFIQs: Face Image Quality Assessment Using Gradient Magnitudes](https://mlsanity.com/paper/2404.12203/) (2024)
- [Quality Measures in Biometric Systems](https://mlsanity.com/paper/2111.08704/) (2021)
- [ViT-FIQA: Assessing Face Image Quality using Vision Transformers](https://mlsanity.com/paper/2508.13957/) (2025)
- [FaceQgen: Semi-Supervised Deep Learning for Face Image Quality Assessment](https://mlsanity.com/paper/2201.00770/) (2022)

## Abstract (arXiv)

Face Image Quality Assessment (FIQA) aims to estimate the utility of facial images for reliable recognition. The evaluation of FIQA methods is predominantly based on the Error-versus-Discard Characteristic (EDC), which evaluates performance by progressively discarding low-quality samples and measuring recognition error on the retained subset. In this work, we demonstrate that the widely used EDC protocol has fundamental limitations: Test-Set Divergence and Threshold Drift, which together limit the reliability and comparability of FIQA methods. To address this, we propose discard-based EDC variants and a rank-based Rank Consistency Evaluation (RCE) metric that operates on the entire test set without discarding samples, using a fixed decision threshold. Extensive experiments on five datasets, four face recognition models, and 15 state-of-the-art FIQA methods demonstrate both the limitations of EDC and the effectiveness of the proposed approaches in enabling a more reliable and comparable evaluation. Despite evaluated on face images only, the limitations arise from the EDC protocol rather than the biometric modality, suggesting a broader applicability to biometric quality assessment in general.