# Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment

arXiv: https://arxiv.org/abs/1901.07694 · HTML: https://mlsanity.com/paper/1901.07694/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2019.

## Built on by (most cited first, 30 of 31)

- [Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations](https://mlsanity.com/paper/2301.07255/) (2023)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [NICE: An Algorithm for Nearest Instance Counterfactual Explanations](https://mlsanity.com/paper/2104.07411/) (2021)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making](https://mlsanity.com/paper/2403.16812/) (2024)
- [Selective Explanations: Leveraging Human Input to Align Explainable AI](https://mlsanity.com/paper/2301.09656/) (2023)
- [Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking](https://mlsanity.com/paper/2502.09083/) (2025)
- [Fairness Perceptions of Algorithmic Decision-Making: A Systematic Review of the Empirical Literature](https://mlsanity.com/paper/2103.12016/) (2021)
- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [Seamful XAI: Operationalizing Seamful Design in Explainable AI](https://mlsanity.com/paper/2211.06753/) (2022)
- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Application](https://mlsanity.com/paper/2305.08598/) (2023)
- [EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations](https://mlsanity.com/paper/2402.00491/) (2024)
- [Towards a multi-stakeholder value-based assessment framework for algorithmic systems](https://mlsanity.com/paper/2205.04525/) (2022)
- [Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ](https://mlsanity.com/paper/2212.08733/) (2022)
- [Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems](https://mlsanity.com/paper/2108.06500/) (2021)
- [Towards Responsible AI: A Design Space Exploration of Human-Centered Artificial Intelligence User Interfaces to Investigate Fairness](https://mlsanity.com/paper/2206.00474/) (2022)
- [IRIS: Interpretable Rubric-Informed Segmentation for Action Quality Assessment](https://mlsanity.com/paper/2303.09097/) (2023)
- [DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction](https://mlsanity.com/paper/2303.01576/) (2023)
- [Use of What-if Scenarios to Help Explain Artificial Intelligence Models for Neonatal Health](https://mlsanity.com/paper/2410.09635/) (2024)
- [FATE in AI: Towards Algorithmic Inclusivity and Accessibility](https://mlsanity.com/paper/2301.01590/) (2023)
- [FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines](https://mlsanity.com/paper/2307.15475/) (2023)
- [The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features](https://mlsanity.com/paper/2310.08617/) (2023)
- [On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2510.12201/) (2025)
- [A Human-Centric Perspective on Fairness and Transparency in Algorithmic Decision-Making](https://mlsanity.com/paper/2205.00033/) (2022)
- [Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development](https://mlsanity.com/paper/2403.15481/) (2024)
- [Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions](https://mlsanity.com/paper/2505.01192/) (2025)
- [Tensions Between the Proxies of Human Values in AI](https://mlsanity.com/paper/2212.07508/) (2022)
- ["Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study](https://mlsanity.com/paper/2504.11020/) (2025)

## Abstract (arXiv)

Ensuring fairness of machine learning systems is a human-in-the-loop process. It relies on developers, users, and the general public to identify fairness problems and make improvements. To facilitate the process we need effective, unbiased, and user-friendly explanations that people can confidently rely on. Towards that end, we conducted an empirical study with four types of programmatically generated explanations to understand how they impact people's fairness judgments of ML systems. With an experiment involving more than 160 Mechanical Turk workers, we show that: 1) Certain explanations are considered inherently less fair, while others can enhance people's confidence in the fairness of the algorithm; 2) Different fairness problems--such as model-wide fairness issues versus case-specific fairness discrepancies--may be more effectively exposed through different styles of explanation; 3) Individual differences, including prior positions and judgment criteria of algorithmic fairness, impact how people react to different styles of explanation. We conclude with a discussion on providing personalized and adaptive explanations to support fairness judgments of ML systems.