# Explainable AI Isn't Enough! Rethinking Algorithmic Contestability

arXiv: https://arxiv.org/abs/2605.16041 · HTML: https://mlsanity.com/paper/2605.16041/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 89 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 24 of 24)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020)
- [Towards Robust and Reliable Algorithmic Recourse](https://mlsanity.com/paper/2102.13620/) (2021)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [On the Fairness of Causal Algorithmic Recourse](https://mlsanity.com/paper/2010.06529/) (2020)
- [Amazing Things Come From Having Many Good Models](https://mlsanity.com/paper/2407.04846/) (2024)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples](https://mlsanity.com/paper/2009.05487/) (2020)
- [shapiq: Shapley Interactions for Machine Learning](https://mlsanity.com/paper/2410.01649/) (2024)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)
- [On the Privacy Risks of Algorithmic Recourse](https://mlsanity.com/paper/2211.05427/) (2022)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Sampling, Intervention, Prediction, Aggregation: A Generalized Framework for Model-Agnostic Interpretations](https://mlsanity.com/paper/1904.03959/) (2019)
- [Perceptions of the Fairness Impacts of Multiplicity in Machine Learning](https://mlsanity.com/paper/2409.12332/) (2024)
- [Contestable Black Boxes](https://mlsanity.com/paper/2006.05133/) (2020)
- [The Role of Hyperparameters in Predictive Multiplicity](https://mlsanity.com/paper/2503.13506/) (2025)
- [Two Means to an End Goal: Connecting Explainability and Contestability in the Regulation of Public Sector AI](https://mlsanity.com/paper/2504.18236/) (2025)
- [Explainable AI Systems Must Be Contestable: Here's How to Make It Happen](https://mlsanity.com/paper/2506.01662/) (2025)

## Abstract (arXiv)

Machine learning systems increasingly make life-changing decisions about individuals, such as loan approvals, hiring, and cheating detection, raising a pressing question: how can individuals respond to negative decisions made by these opaque systems? While explainable artificial intelligence (XAI) has largely focused on algorithmic recourse -- helping individuals change their features to obtain a desired outcome -- the parallel problem of algorithmic contestability -- helping individuals review and correct erroneous algorithmic decisions -- has received far less attention, despite its central ethical and legal importance. We trace this neglect to the absence of clear formal definitions and a systematic operationalization of contestability as an algorithmic problem. To address it, we propose an operational definition of contestability as a natural complement to recourse: contestability starts from the presumption that a decision may be incorrect and focuses on identifying evidence to challenge and potentially overturn it, whereas recourse assumes the decision is valid and instead provides pathways for changing it. We show that standard XAI explanations, such as counterfactuals, LIME, or Anchors, even when combined with human intuitions about decision continuity or monotonicity, reveal only errors in the neighborhood of the individual, but provide insufficient grounds for overturning the decision at hand. Going thus beyond traditional XAI, we identify three types of evidence warranting reversal according to the decision maker's own ethical standards: predictive multiplicity, incorrect feature values, and neglected overruling evidence. We argue that these render decisions normatively indefensible and thus successfully contestable. Finally, we analyze how existing EU legislation connects to our framework and argue that individuals already hold some legal rights to these forms of evidence.