# Choose your Data Wisely: A Framework for Semantic Counterfactuals

arXiv: https://arxiv.org/abs/2305.17667 · HTML: https://mlsanity.com/paper/2305.17667/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 28 May 2023.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [RelTR: Relation Transformer for Scene Graph Generation](https://mlsanity.com/paper/2201.11460/) (2022)
- [Coswara -- A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis](https://mlsanity.com/paper/2005.10548/) (2020)
- [Making Heads or Tails: Towards Semantically Consistent Visual Counterfactuals](https://mlsanity.com/paper/2203.12892/) (2022)
- [Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networks](https://mlsanity.com/paper/2012.10076/) (2020)

## Built on by (most cited first, 7 of 7)

- [Even-if Explanations: Formal Foundations, Priorities and Complexity](https://mlsanity.com/paper/2401.10938/) (2024)
- [V-CECE: Visual Counterfactual Explanations via Conceptual Edits](https://mlsanity.com/paper/2509.16567/) (2025)
- [Semantic Prototypes: Enhancing Transparency Without Black Boxes](https://mlsanity.com/paper/2407.15871/) (2024)
- [Graph Edits for Counterfactual Explanations: A comparative study](https://mlsanity.com/paper/2401.11609/) (2024)
- [Beyond One-Size-Fits-All: Adapting Counterfactual Explanations to User Objectives](https://mlsanity.com/paper/2404.08721/) (2024)
- [Semantic Prioritization in Visual Counterfactual Explanations with Weighted Segmentation and Auto-Adaptive Region Selection](https://mlsanity.com/paper/2511.12992/) (2025)
- [U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanations](https://mlsanity.com/paper/2604.08295/) (2026)

## Abstract (arXiv)

Counterfactual explanations have been argued to be one of the most intuitive forms of explanation. They are typically defined as a minimal set of edits on a given data sample that, when applied, changes the output of a model on that sample. However, a minimal set of edits is not always clear and understandable to an end-user, as it could, for instance, constitute an adversarial example (which is indistinguishable from the original data sample to an end-user). Instead, there are recent ideas that the notion of minimality in the context of counterfactuals should refer to the semantics of the data sample, and not to the feature space. In this work, we build on these ideas, and propose a framework that provides counterfactual explanations in terms of knowledge graphs. We provide an algorithm for computing such explanations (given some assumptions about the underlying knowledge), and quantitatively evaluate the framework with a user study.