# The Conflict Between Explainable and Accountable Decision-Making Algorithms

arXiv: https://arxiv.org/abs/2205.05306 · HTML: https://mlsanity.com/paper/2205.05306/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 108 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 24 of 24)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [Principles alone cannot guarantee ethical AI](https://mlsanity.com/paper/1906.06668/) (2019)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [Artificial Intelligence: the global landscape of ethics guidelines](https://mlsanity.com/paper/1906.11668/) (2019)
- [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021)
- [Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination](https://mlsanity.com/paper/1408.6491/) (2014)
- [The Use and Misuse of Counterfactuals in Ethical Machine Learning](https://mlsanity.com/paper/2102.05085/) (2021)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [Explainability Pitfalls: Beyond Dark Patterns in Explainable AI](https://mlsanity.com/paper/2109.12480/) (2021)
- [Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems](https://mlsanity.com/paper/2102.04201/) (2021)
- [Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems](https://mlsanity.com/paper/2101.09385/) (2021)
- [Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making](https://mlsanity.com/paper/2102.00625/) (2021)
- [The Conflict Between People's Urge to Punish AI and Legal Systems](https://mlsanity.com/paper/2003.06507/) (2020)

## Built on by (most cited first, 13 of 13)

- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Algorithmic Assistance with Recommendation-Dependent Preferences](https://mlsanity.com/paper/2208.07626/) (2022)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions](https://mlsanity.com/paper/2503.00940/) (2025)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [Blaming Humans and Machines: What Shapes People's Reactions to Algorithmic Harm](https://mlsanity.com/paper/2304.02176/) (2023)
- [SoUnD Framework: Analyzing (So)cial Representation in (Un)structured (D)ata](https://mlsanity.com/paper/2311.17259/) (2023)
- [Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI Research](https://mlsanity.com/paper/2312.12119/) (2023)
- [From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap](https://mlsanity.com/paper/2404.13131/) (2024)
- [Being Accountable is Smart: Navigating the Technical and Regulatory Landscape of AI-based Services for Power Grid](https://mlsanity.com/paper/2408.01121/) (2024)
- [Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifiers](https://mlsanity.com/paper/2412.03166/) (2024)
- [Onto-Epistemological Analysis of AI Explanations](https://mlsanity.com/paper/2510.02996/) (2025)

## Abstract (arXiv)

Decision-making algorithms are being used in important decisions, such as who should be enrolled in health care programs and be hired. Even though these systems are currently deployed in high-stakes scenarios, many of them cannot explain their decisions. This limitation has prompted the Explainable Artificial Intelligence (XAI) initiative, which aims to make algorithms explainable to comply with legal requirements, promote trust, and maintain accountability. This paper questions whether and to what extent explainability can help solve the responsibility issues posed by autonomous AI systems. We suggest that XAI systems that provide post-hoc explanations could be seen as blameworthy agents, obscuring the responsibility of developers in the decision-making process. Furthermore, we argue that XAI could result in incorrect attributions of responsibility to vulnerable stakeholders, such as those who are subjected to algorithmic decisions (i.e., patients), due to a misguided perception that they have control over explainable algorithms. This conflict between explainability and accountability can be exacerbated if designers choose to use algorithms and patients as moral and legal scapegoats. We conclude with a set of recommendations for how to approach this tension in the socio-technical process of algorithmic decision-making and a defense of hard regulation to prevent designers from escaping responsibility.