mlsanityData to 7 Oct 2026

Connecting Algorithmic Fairness to Quality Dimensions in Machine Learning in Official Statistics and Survey Production

arXiv 2402.09328 ↗First posted 14 Feb 2024stat.ML cs.LG stat.MEcited by 0

Not on the map. Why?

Abstract (arXiv)

National Statistical Organizations (NSOs) increasingly draw on Machine Learning (ML) to improve the timeliness and cost-effectiveness of their products. When introducing ML solutions, NSOs must ensure that high standards with respect to robustness, reproducibility, and accuracy are upheld as codified, e.g., in the Quality Framework for Statistical Algorithms (QF4SA; Yung et al. 2022). At the same time, a growing body of research focuses on fairness as a pre-condition of a safe deployment of ML to prevent disparate social impacts in practice. However, fairness has not yet been explicitly discussed as a quality aspect in the context of the application of ML at NSOs. We employ Yung et al. (2022)'s QF4SA quality framework and present a mapping of its quality dimensions to algorithmic fairness. We thereby extend the QF4SA framework in several ways: we argue for fairness as its own quality dimension, we investigate the interaction of fairness with other dimensions, and we explicitly address data, both on its own and its interaction with applied methodology. In parallel with empirical illustrations, we show how our mapping can contribute to methodology in the domains of official statistics, algorithmic fairness, and trustworthy machine learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 157 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

47 of 157 references.

References · 47

  1. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  2. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  4. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  5. 2018Model Cards for Model Reportingcited by 1,068
  6. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  7. 2017Counterfactual Fairnesscited by 802
  8. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  9. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  10. 2016The Mythos of Model Interpretabilitycited by 716
  11. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  12. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
35 more
  1. 2020Fairness in Machine Learning: A Surveycited by 369
  2. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  3. 2015Recursive Partitioning for Heterogeneous Causal Effectscited by 254
  4. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  5. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
  6. 2017Avoiding Discrimination through Causal Reasoningcited by 191
  7. 2020Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Reviewcited by 174
  8. 2020Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scanscited by 158
  9. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
  10. 2017Fairness in Machine Learning: Lessons from Political Philosophycited by 148
  11. 2018Aequitas: A Bias and Fairness Audit Toolkitcited by 145
  12. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  13. 2020A survey of algorithmic recourse: definitions, formulations, solutions, and prospectscited by 118
  14. 2021Algorithmic Monoculture and Social Welfarecited by 98
  15. 2017On Formalizing Fairness in Prediction with Machine Learningcited by 65
  16. 2022Leakage and the Reproducibility Crisis in ML-based Sciencecited by 63
  17. 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 60
  18. 2019Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learningcited by 57
  19. 2020Survey on Causal-based Machine Learning Fairness Notionscited by 53
  20. 2022Rethinking Explainability as a Dialogue: A Practitioner's Perspectivecited by 52
  21. 2022Robust Counterfactual Explanations for Tree-Based Ensemblescited by 49
  22. 2018Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Predictioncited by 35
  23. 2024The illusion of artificial inclusioncited by 34
  24. 2021Unraveling the Hidden Environmental Impacts of AI Solutions for Environmentcited by 32
  25. 2021Fairness Perceptions of Algorithmic Decision-Making: A Systematic Review of the Empirical Literaturecited by 27
  26. 2021Towards Green Automated Machine Learning: Status Quo and Future Directionscited by 25
  27. 2020Private Post-GAN Boostingcited by 20
  28. 2023Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoMLcited by 16
  29. 2020A Methodology for Creating AI FactSheetscited by 15
  30. 2022Data Representativity for Machine Learning and AI Systemscited by 15
  31. 2023One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisionscited by 12
  32. 2023Automated wildlife image classification: An active learning tool for ecological applicationscited by 11
  33. 2021Fairness in Algorithmic Profiling: A German Case Studycited by 6
  34. 2022Uncertainty-aware predictive modeling for fair data-driven decisionscited by 6
  35. 2022Locating disparities in machine learningcited by 4

Built on by

Not cited yet.