# Five policy uses of algorithmic transparency and explainability

arXiv: https://arxiv.org/abs/2302.03080 · HTML: https://mlsanity.com/paper/2302.03080/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 139 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- ["How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations](https://mlsanity.com/paper/1911.06473/) (2019)
- [Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences](https://mlsanity.com/paper/1712.00547/) (2017)
- [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021)
- [Fairwashing: the risk of rationalization](https://mlsanity.com/paper/1901.09749/) (2019)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems](https://mlsanity.com/paper/1806.07552/) (2018)
- [Order in the Court: Explainable AI Methods Prone to Disagreement](https://mlsanity.com/paper/2105.03287/) (2021)
- [Principles of Explanation in Human-AI Systems](https://mlsanity.com/paper/2102.04972/) (2021)
- [Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination](https://mlsanity.com/paper/1912.06171/) (2019)
- [Machine Learning Explainability for External Stakeholders](https://mlsanity.com/paper/2007.05408/) (2020)
- [Explainable Machine Learning for Public Policy: Use Cases, Gaps, and Research Directions](https://mlsanity.com/paper/2010.14374/) (2020)
- [Seven challenges for harmonizing explainability requirements](https://mlsanity.com/paper/2108.05390/) (2021)

## Abstract (arXiv)

The notion that algorithmic systems should be "transparent" and "explainable" is common in the many statements of consensus principles developed by governments, companies, and advocacy organizations. But what exactly do policy and legal actors want from these technical concepts, and how do their desiderata compare with the explainability techniques developed in the machine learning literature? In hopes of better connecting the policy and technical communities, we provide case studies illustrating five ways in which algorithmic transparency and explainability have been used in policy settings: specific requirements for explanations; in nonbinding guidelines for internal governance of algorithms; in regulations applicable to highly regulated settings; in guidelines meant to increase the utility of legal liability for algorithms; and broad requirements for model and data transparency. The case studies span a spectrum from precise requirements for specific types of explanations to nonspecific requirements focused on broader notions of transparency, illustrating the diverse needs, constraints, and capacities of various policy actors and contexts. Drawing on these case studies, we discuss promising ways in which transparency and explanation could be used in policy, as well as common factors limiting policymakers' use of algorithmic explainability. We conclude with recommendations for researchers and policymakers.