# "If it didn't happen, why would I change my decision?": How Judges Respond to Counterfactual Explanations for the Public Safety Assessment

arXiv: https://arxiv.org/abs/2205.05424 · HTML: https://mlsanity.com/paper/2205.05424/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2022.

## Built on by (most cited first, 6 of 6)

- [Towards Conceptualization of "Fair Explanation": Disparate Impacts of anti-Asian Hate Speech Explanations on Content Moderators](https://mlsanity.com/paper/2310.15055/) (2023)
- [Reassessing Evaluation Functions in Algorithmic Recourse: An Empirical Study from a Human-Centered Perspective](https://mlsanity.com/paper/2405.14264/) (2024)
- [T-COL: Generating Counterfactual Explanations for General User Preferences on Variable Machine Learning Systems](https://mlsanity.com/paper/2309.16146/) (2023)
- [Tensions Between the Proxies of Human Values in AI](https://mlsanity.com/paper/2212.07508/) (2022)
- [Mind the Gap: Optimal and Equitable Encouragement Policies](https://mlsanity.com/paper/2309.07176/) (2023)
- [What Do People Want to Know About Artificial Intelligence (AI)? The Importance of Answering End-User Questions to Explain Autonomous Vehicle (AV) Decisions](https://mlsanity.com/paper/2505.06428/) (2025)

## Abstract (arXiv)

Many researchers and policymakers have expressed excitement about algorithmic explanations enabling more fair and responsible decision-making. However, recent experimental studies have found that explanations do not always improve human use of algorithmic advice. In this study, we shed light on how people interpret and respond to counterfactual explanations (CFEs) -- explanations that show how a model's output would change with marginal changes to its input(s) -- in the context of pretrial risk assessment instruments (PRAIs). We ran think-aloud trials with eight sitting U.S. state court judges, providing them with recommendations from a PRAI that includes CFEs. We found that the CFEs did not alter the judges' decisions. At first, judges misinterpreted the counterfactuals as real -- rather than hypothetical -- changes to defendants. Once judges understood what the counterfactuals meant, they ignored them, stating their role is only to make decisions regarding the actual defendant in question. The judges also expressed a mix of reasons for ignoring or following the advice of the PRAI without CFEs. These results add to the literature detailing the unexpected ways in which people respond to algorithms and explanations. They also highlight new challenges associated with improving human-algorithm collaborations through explanations.