# Counterfactual Explanation of Brain Activity Classifiers using Image-to-Image Transfer by Generative Adversarial Network

arXiv: https://arxiv.org/abs/2110.14927 · HTML: https://mlsanity.com/paper/2110.14927/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [Efficient Decision-based Black-box Adversarial Attacks on Face Recognition](https://mlsanity.com/paper/1904.04433/) (2019)
- [Explaining Image Classifiers by Counterfactual Generation](https://mlsanity.com/paper/1807.08024/) (2018)
- [Explanation by Progressive Exaggeration](https://mlsanity.com/paper/1911.00483/) (2019)
- [SCOUT: Self-aware Discriminant Counterfactual Explanations](https://mlsanity.com/paper/2004.07769/) (2020)
- [GANterfactual - Counterfactual Explanations for Medical Non-Experts using Generative Adversarial Learning](https://mlsanity.com/paper/2012.11905/) (2020)
- [Generative Counterfactual Introspection for Explainable Deep Learning](https://mlsanity.com/paper/1907.03077/) (2019)
- [Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer's disease classification](https://mlsanity.com/paper/1909.08856/) (2019)
- [Decoding and mapping task states of the human brain via deep learning](https://mlsanity.com/paper/1801.09858/) (2018)
- [Scientific Discovery by Generating Counterfactuals using Image Translation](https://mlsanity.com/paper/2007.05500/) (2020)
- [Fast Real-time Counterfactual Explanations](https://mlsanity.com/paper/2007.05684/) (2020)

## Built on by (most cited first, 5 of 5)

- [GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanations](https://mlsanity.com/paper/2302.12689/) (2023)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [Robust image representations with counterfactual contrastive learning](https://mlsanity.com/paper/2409.10365/) (2024)
- [Counterfactual contrastive learning: robust representations via causal image synthesis](https://mlsanity.com/paper/2403.09605/) (2024)
- [A Survey on Interpretability in Visual Recognition](https://mlsanity.com/paper/2507.11099/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the nonlinearity of the DNN, the decisions made by DNNs are hardly interpretable. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. Here we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among multiple classes associated with different behavioral tasks. Using CAG, we demonstrated counterfactual explanation of DNN-based classifiers that learned to discriminate brain activations of seven behavioral tasks. Furthermore, by iterative applications of CAG, we were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses.