# RecourseBench: A Modular Framework for Reproducible Algorithmic Recourse Evaluation

arXiv: https://arxiv.org/abs/2606.16113 · HTML: https://mlsanity.com/paper/2606.16113/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 15 Jun 2026.

## Builds on (most cited first, 30 of 35)

- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [Learning Model-Agnostic Counterfactual Explanations for Tabular Data](https://mlsanity.com/paper/1910.09398/) (2019)
- [Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers](https://mlsanity.com/paper/1912.03277/) (2019)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [Towards Robust and Reliable Algorithmic Recourse](https://mlsanity.com/paper/2102.13620/) (2021)
- [Getting a CLUE: A Method for Explaining Uncertainty Estimates](https://mlsanity.com/paper/2006.06848/) (2020)
- [CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms](https://mlsanity.com/paper/2108.00783/) (2021)
- [Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking](https://mlsanity.com/paper/1706.06691/) (2017)
- [Inverse Classification for Comparison-based Interpretability in Machine Learning](https://mlsanity.com/paper/1712.08443/) (2017)
- [Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse](https://mlsanity.com/paper/2203.06768/) (2022)
- [Consistent Counterfactuals for Deep Models](https://mlsanity.com/paper/2110.03109/) (2021)
- [Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers](https://mlsanity.com/paper/2406.14325/) (2024)
- [On the Robustness of Sparse Counterfactual Explanations to Adverse Perturbations](https://mlsanity.com/paper/2201.09051/) (2022)
- [Provably Robust and Plausible Counterfactual Explanations for Neural Networks via Robust Optimisation](https://mlsanity.com/paper/2309.12545/) (2023)
- [Robust Bayesian Recourse](https://mlsanity.com/paper/2206.10833/) (2022)
- [Promoting Counterfactual Robustness through Diversity](https://mlsanity.com/paper/2312.06564/) (2023)
- [Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning](https://mlsanity.com/paper/2106.02597/) (2021)
- [What is Reproducibility in Artificial Intelligence and Machine Learning Research?](https://mlsanity.com/paper/2407.10239/) (2024)
- [Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions](https://mlsanity.com/paper/2111.01235/) (2021)
- [Endogenous Macrodynamics in Algorithmic Recourse](https://mlsanity.com/paper/2308.08187/) (2023)
- [Counterfactual Explanations via Riemannian Latent Space Traversal](https://mlsanity.com/paper/2411.02259/) (2024)
- [Flexible and Robust Counterfactual Explanations with Minimal Satisfiable Perturbations](https://mlsanity.com/paper/2309.04676/) (2023)
- [RobustX: Robust Counterfactual Explanations Made Easy](https://mlsanity.com/paper/2502.13751/) (2025)
- [Explaining Black-Box Models through Counterfactuals](https://mlsanity.com/paper/2308.07198/) (2023)

## Abstract (arXiv)

Algorithmic recourse methods provide counterfactual explanations that inform individuals of the actions required to overturn an unfavorable model decision. Despite rapid methodological progress, principled comparison remains elusive; existing frameworks are often difficult to extend and lack both interoperability and systematic verification that integrated methods faithfully reproduce their originally reported claims. We introduce RecourseBench, a unified evaluation framework built around three commitments: modularity, reproducibility, and interactivity. The framework decomposes the pipeline into five fully decoupled layers---Data, Preprocessing, Model, Recourse Method, and Evaluation---governed by abstract interfaces and a dynamic registry. Every integrated method is classified into a four-tier reproducibility taxonomy based on artifact availability, followed by a systematic verification of its core empirical claims. We further provide an interactive web interface for flexible, configuration-driven exploration across datasets, model architectures, methods, and evaluations. To our knowledge, RecourseBench is the first benchmark to explicitly ground recourse evaluation in structured claim verification and mathematically rigorous reproducibility standards, all while featuring the largest collection of state-of-the-art recourse algorithms (27 in total).