mlsanityData to 7 Oct 2026

Beyond Fairness Metrics: Roadblocks and Challenges for Ethical AI in Practice

arXiv 2108.06217 ↗First posted 11 Aug 2021cs.LG cs.AI cs.CYcited by 4

Not on the map. Why?

Abstract (arXiv)

We review practical challenges in building and deploying ethical AI at the scale of contemporary industrial and societal uses. Apart from the purely technical concerns that are the usual focus of academic research, the operational challenges of inconsistent regulatory pressures, conflicting business goals, data quality issues, development processes, systems integration practices, and the scale of deployment all conspire to create new ethical risks. Such ethical concerns arising from these practical considerations are not adequately addressed by existing research results. We argue that a holistic consideration of ethics in the development and deployment of AI systems is necessary for building ethical AI in practice, and exhort researchers to consider the full operational contexts of AI systems when assessing ethical risks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 22 of its 85 references that are in the map. First posted on arXiv 11 Aug 2021.

Builds on

22 of 85 references.

References · 22

  1. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  2. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  3. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  4. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  5. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  6. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  7. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  8. 2020Fairness in Machine Learningcited by 194
  9. 2020Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scanscited by 158
  10. 2019Principles alone cannot guarantee ethical AIcited by 134
  11. 2019The Ethics of AI Ethics -- An Evaluation of Guidelinescited by 129
  12. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
10 more
  1. 2020Estimating Example Difficulty Using Variance of Gradientscited by 96
  2. 2019Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combinationcited by 79
  3. 2020Decisions, Counterfactual Explanations and Strategic Behaviorcited by 44
  4. 2020The Grey Hoodie Project: Big Tobacco, Big Tech, and the threat on academic integritycited by 38
  5. 2021Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Informationcited by 33
  6. 2019Regulatory Markets for AI Safetycited by 13
  7. 2020Debiasing classifiers: is reality at variance with expectation?cited by 10
  8. 2020Towards Self-Regulating AI: Challenges and Opportunities of AI Model Governance in Financial Servicescited by 8
  9. 2020Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Societycited by 7
  10. 2020Towards organizational guidelines for the responsible use of AIcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 1.

4 papers.

  1. 2023Users are the North Star for AI Transparencycited by 5
  2. 2022Beyond Mahalanobis-Based Scores for Textual OOD Detectioncited by 3
  3. 2025Talking About the Assumption in the Roomcited by 2
  4. 2024Artificial Intelligence and Diabetes Mellitus: An Inside Look Through the Retinacited by 0