Connecting Algorithmic Fairness to Quality Dimensions in Machine Learning in Official Statistics and Survey Production
Not on the map. Why?
Abstract (arXiv)
National Statistical Organizations (NSOs) increasingly draw on Machine Learning (ML) to improve the timeliness and cost-effectiveness of their products. When introducing ML solutions, NSOs must ensure that high standards with respect to robustness, reproducibility, and accuracy are upheld as codified, e.g., in the Quality Framework for Statistical Algorithms (QF4SA; Yung et al. 2022). At the same time, a growing body of research focuses on fairness as a pre-condition of a safe deployment of ML to prevent disparate social impacts in practice. However, fairness has not yet been explicitly discussed as a quality aspect in the context of the application of ML at NSOs. We employ Yung et al. (2022)'s QF4SA quality framework and present a mapping of its quality dimensions to algorithmic fairness. We thereby extend the QF4SA framework in several ways: we argue for fairness as its own quality dimension, we investigate the interaction of fairness with other dimensions, and we explicitly address data, both on its own and its interaction with applied methodology. In parallel with empirical illustrations, we show how our mapping can contribute to methodology in the domains of official statistics, algorithmic fairness, and trustworthy machine learning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 157 references that are in the map. First posted on arXiv 14 Feb 2024.
Builds on
47 of 157 references.
References · 47
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
- 2018Model Cards for Model Reportingcited by 1,068
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
- 2017Counterfactual Fairnesscited by 802
- 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
- 2016The Mythos of Model Interpretabilitycited by 716
- 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
- 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
35 more
- 2020Fairness in Machine Learning: A Surveycited by 369
- 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
- 2015Recursive Partitioning for Heterogeneous Causal Effectscited by 254
- 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
- 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
- 2017Avoiding Discrimination through Causal Reasoningcited by 191
- 2020Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Reviewcited by 174
- 2020Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scanscited by 158
- 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
- 2017Fairness in Machine Learning: Lessons from Political Philosophycited by 148
- 2018Aequitas: A Bias and Fairness Audit Toolkitcited by 145
- 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
- 2020A survey of algorithmic recourse: definitions, formulations, solutions, and prospectscited by 118
- 2021Algorithmic Monoculture and Social Welfarecited by 98
- 2017On Formalizing Fairness in Prediction with Machine Learningcited by 65
- 2022Leakage and the Reproducibility Crisis in ML-based Sciencecited by 63
- 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 60
- 2019Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learningcited by 57
- 2020Survey on Causal-based Machine Learning Fairness Notionscited by 53
- 2022Rethinking Explainability as a Dialogue: A Practitioner's Perspectivecited by 52
- 2022Robust Counterfactual Explanations for Tree-Based Ensemblescited by 49
- 2018Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Predictioncited by 35
- 2024The illusion of artificial inclusioncited by 34
- 2021Unraveling the Hidden Environmental Impacts of AI Solutions for Environmentcited by 32
- 2021Fairness Perceptions of Algorithmic Decision-Making: A Systematic Review of the Empirical Literaturecited by 27
- 2021Towards Green Automated Machine Learning: Status Quo and Future Directionscited by 25
- 2020Private Post-GAN Boostingcited by 20
- 2023Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoMLcited by 16
- 2020A Methodology for Creating AI FactSheetscited by 15
- 2022Data Representativity for Machine Learning and AI Systemscited by 15
- 2023One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisionscited by 12
- 2023Automated wildlife image classification: An active learning tool for ecological applicationscited by 11
- 2021Fairness in Algorithmic Profiling: A German Case Studycited by 6
- 2022Uncertainty-aware predictive modeling for fair data-driven decisionscited by 6
- 2022Locating disparities in machine learningcited by 4
Built on by
Not cited yet.