mlsanityData to 7 Oct 2026

Measuring Bias in AI Models: An Statistical Approach Introducing N-Sigma

arXiv 2304.13680 ↗First posted 26 Apr 2023cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

The new regulatory framework proposal on Artificial Intelligence (AI) published by the European Commission establishes a new risk-based legal approach. The proposal highlights the need to develop adequate risk assessments for the different uses of AI. This risk assessment should address, among others, the detection and mitigation of bias in AI. In this work we analyze statistical approaches to measure biases in automatic decision-making systems. We focus our experiments in face recognition technologies. We propose a novel way to measure the biases in machine learning models using a statistical approach based on the N-Sigma method. N-Sigma is a popular statistical approach used to validate hypotheses in general science such as physics and social areas and its application to machine learning is yet unexplored. In this work we study how to apply this methodology to develop new risk assessment frameworks based on bias analysis and we discuss the main advantages and drawbacks with respect to other popular statistical tests.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 26 Apr 2023.

Builds on

21 of 41 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  3. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  5. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 621
  6. 2020Fairness in Machine Learningcited by 194
  7. 2019Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learningcited by 145
  8. 2019On the Fairness of Disentangled Representationscited by 144
  9. 2020Demographic Bias in Biometrics: A Survey on an Emerging Challengecited by 93
  10. 2021A Comprehensive Study on Face Recognition Biases Beyond Demographicscited by 86
  11. 2020Face Image Quality Assessment: A Literature Surveycited by 79
  12. 2020Mitigating Face Recognition Bias via Group Adaptive Classifiercited by 67
9 more
  1. 2020SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learningcited by 50
  2. 2020Fairness in Biometrics: a figure of merit to assess biometric verification systemscited by 41
  3. 2020InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometricscited by 31
  4. 2018An Experimental Evaluation of Covariates Effects on Unconstrained Face Verificationcited by 28
  5. 2019Deep Learning for Face Recognition: Pride or Prejudiced?cited by 25
  6. 2022Facial Soft Biometrics for Recognition in the Wild: Recent Works, Annotation, and COTS Evaluationcited by 19
  7. 2021SetMargin Loss applied to Deep Keystroke Biometrics with Circle Packing Interpretationcited by 11
  8. 2021FaceQvec: Vector Quality Assessment for Face Biometrics based on ISO Compliancecited by 8
  9. 2022On Demographic Bias in Fingerprint Recognitioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 5; 2026: 3.

11 papers.

  1. 2023Leveraging Large Language Models for Topic Classification in the Domain of Public Affairscited by 13
  2. 2025Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitmentcited by 12
  3. 2025PBa-LLM: Privacy- and Bias-aware NLP using Named-Entity Recognition (NER)cited by 10
  4. 2025Position: Bridge the Gaps between Machine Unlearning and AI Regulationcited by 5
  5. 2026Auditing Training Data in Domain-adapted LLMs: LoRA-MINTcited by 3
  6. 2023Document Layout Annotation: Database and Benchmark in the Domain of Public Affairscited by 2
  7. 2024Comprehensive Equity Index (CEI): Definition and Application to Bias Evaluation in Biometricscited by 2
  8. 2025Active Membership Inference Test (aMINT): Enhancing Model Auditability with Multi-Task Learningcited by 2
  9. 2026Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Visioncited by 2
  10. 2025Downsized and Compromised?: Assessing the Faithfulness of Model Compressioncited by 0
  11. 2026Measures of classification bias derived from sample size analysiscited by 0