# Counterfactual Explanations for Arbitrary Regression Models

arXiv: https://arxiv.org/abs/2106.15212 · HTML: https://mlsanity.com/paper/2106.15212/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 68 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [Interpretable Counterfactual Explanations Guided by Prototypes](https://mlsanity.com/paper/1907.02584/) (2019)
- [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020)
- [Learning Model-Agnostic Counterfactual Explanations for Tabular Data](https://mlsanity.com/paper/1910.09398/) (2019)
- [Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers](https://mlsanity.com/paper/1912.03277/) (2019)
- [Non-convex Optimization for Machine Learning](https://mlsanity.com/paper/1712.07897/) (2017)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking](https://mlsanity.com/paper/1706.06691/) (2017)
- [On Counterfactual Explanations under Predictive Multiplicity](https://mlsanity.com/paper/2006.13132/) (2020)
- [Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables](https://mlsanity.com/paper/1910.01688/) (2019)
- [Bayesian Optimisation over Multiple Continuous and Categorical Inputs](https://mlsanity.com/paper/1906.08878/) (2019)
- [Bayesian Optimization of Composite Functions](https://mlsanity.com/paper/1906.01537/) (2019)
- [Decisions, Counterfactual Explanations and Strategic Behavior](https://mlsanity.com/paper/2002.04333/) (2020)
- [FOCUS: Flexible Optimizable Counterfactual Explanations for Tree Ensembles](https://mlsanity.com/paper/1911.12199/) (2019)
- [On the computation of counterfactual explanations -- A survey](https://mlsanity.com/paper/1911.07749/) (2019)
- [Convex Density Constraints for Computing Plausible Counterfactual Explanations](https://mlsanity.com/paper/2002.04862/) (2020)
- [Mixed-Variable Bayesian Optimization](https://mlsanity.com/paper/1907.01329/) (2019)
- [The Complexity of Gradient Descent: CLS = PPAD cap PLS](https://mlsanity.com/paper/2011.01929/) (2020)
- [Fair lending needs explainable models for responsible recommendation](https://mlsanity.com/paper/1809.04684/) (2018)

## Built on by (most cited first, 22 of 22)

- [Robust Counterfactual Explanations for Tree-Based Ensembles](https://mlsanity.com/paper/2207.02739/) (2022)
- [Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse](https://mlsanity.com/paper/2203.06768/) (2022)
- [On the Privacy Risks of Algorithmic Recourse](https://mlsanity.com/paper/2211.05427/) (2022)
- [Toward Explainable AI for Regression Models](https://mlsanity.com/paper/2112.11407/) (2021)
- [ReLAX: Reinforcement Learning Agent eXplainer for Arbitrary Predictive Models](https://mlsanity.com/paper/2110.11960/) (2021)
- [Counterfactual Shapley Additive Explanations](https://mlsanity.com/paper/2110.14270/) (2021)
- [On the Trade-Off between Actionable Explanations and the Right to be Forgotten](https://mlsanity.com/paper/2208.14137/) (2022)
- [Counterfactual Metarules for Local and Global Recourse](https://mlsanity.com/paper/2405.18875/) (2024)
- [Mitigating Clever Hans Strategies in Image Classifiers through Generating Counterexamples](https://mlsanity.com/paper/2510.17524/) (2025)
- [Faithful Model Explanations through Energy-Constrained Conformal Counterfactuals](https://mlsanity.com/paper/2312.10648/) (2023)
- [Diffusion Counterfactuals for Image Regressors](https://mlsanity.com/paper/2503.20595/) (2025)
- [Counterfactual Explanations for Deep Learning-Based Traffic Forecasting](https://mlsanity.com/paper/2405.00456/) (2024)
- [Counterfactual Explanations of Neural Network-Generated Response Curves](https://mlsanity.com/paper/2304.04063/) (2023)
- [Unveiling the Potential of Counterfactuals Explanations in Employability](https://mlsanity.com/paper/2305.10069/) (2023)
- [Counterfactual Explanation for Regression via Disentanglement in Latent Space](https://mlsanity.com/paper/2311.08228/) (2023)
- [Decomposing Counterfactual Explanations for Consequential Decision Making](https://mlsanity.com/paper/2211.02151/) (2022)
- [On the Connection between Game-Theoretic Feature Attributions and Counterfactual Explanations](https://mlsanity.com/paper/2307.06941/) (2023)
- [No Dtrain: Model-Agnostic Counterfactual Explanations Using Reinforcement Learning](https://mlsanity.com/paper/2405.18563/) (2024)
- [Relevance-aware Algorithmic Recourse](https://mlsanity.com/paper/2405.19072/) (2024)
- [ACE: Adapting sampling for Counterfactual Explanations](https://mlsanity.com/paper/2509.26322/) (2025)
- [Counterfactual Training: Teaching Models Plausible and Actionable Explanations](https://mlsanity.com/paper/2601.16205/) (2026)
- [CEL: Comprehensive Counterfactual Explanations Library and Benchmark](https://mlsanity.com/paper/2607.22045/) (2026)

## Abstract (arXiv)

We present a new method for counterfactual explanations (CFEs) based on Bayesian optimisation that applies to both classification and regression models. Our method is a globally convergent search algorithm with support for arbitrary regression models and constraints like feature sparsity and actionable recourse, and furthermore can answer multiple counterfactual questions in parallel while learning from previous queries. We formulate CFE search for regression models in a rigorous mathematical framework using differentiable potentials, which resolves robustness issues in threshold-based objectives. We prove that in this framework, (a) verifying the existence of counterfactuals is NP-complete; and (b) that finding instances using such potentials is CLS-complete. We describe a unified algorithm for CFEs using a specialised acquisition function that composes both expected improvement and an exponential-polynomial (EP) family with desirable properties. Our evaluation on real-world benchmark domains demonstrate high sample-efficiency and precision.