# Counterfactual Contrastive Analysis

arXiv: https://arxiv.org/abs/2608.19032 · HTML: https://mlsanity.com/paper/2608.19032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 30 of 30)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021)
- [Diffusion Visual Counterfactual Explanations](https://mlsanity.com/paper/2210.11841/) (2022)
- [Diffusion Models for Counterfactual Explanations](https://mlsanity.com/paper/2203.15636/) (2022)
- [Explanation by Progressive Exaggeration](https://mlsanity.com/paper/1911.00483/) (2019)
- [Image-to-image translation for cross-domain disentanglement](https://mlsanity.com/paper/1805.09730/) (2018)
- [Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations](https://mlsanity.com/paper/2103.10226/) (2021)
- [Learning Disentangled Representations via Mutual Information Estimation](https://mlsanity.com/paper/1912.03915/) (2019)
- [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023)
- [Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation](https://mlsanity.com/paper/2312.14223/) (2023)
- [From Statistical to Causal Learning](https://mlsanity.com/paper/2204.00607/) (2022)
- [Text-to-Image Models for Counterfactual Explanations: a Black-Box Approach](https://mlsanity.com/paper/2309.07944/) (2023)
- [Domain Intersection and Domain Difference](https://mlsanity.com/paper/1908.11628/) (2019)
- [Unsupervised learning with contrastive latent variable models](https://mlsanity.com/paper/1811.06094/) (2018)
- [Moment Matching Deep Contrastive Latent Variable Models](https://mlsanity.com/paper/2202.10560/) (2022)
- [Gacs-Korner Common Information Variational Autoencoder](https://mlsanity.com/paper/2205.12239/) (2022)
- [SepVAE: a contrastive VAE to separate pathological patterns from healthy ones](https://mlsanity.com/paper/2307.06206/) (2023)
- [Double InfoGAN for Contrastive Analysis](https://mlsanity.com/paper/2401.17776/) (2024)
- [Separating common from salient patterns with Contrastive Representation Learning](https://mlsanity.com/paper/2402.11928/) (2024)
- [Learning Common and Salient Generative Factors Between Two Image Datasets](https://mlsanity.com/paper/2512.12800/) (2025)

## Abstract (arXiv)

Visual Counterfactual Explanations (VCEs) aim to explain image classifiers by generating minimally edited and realistic versions of an input image that change the classifier's prediction. Existing VCE methods are inherently classifier-dependent and therefore susceptible to classifier biases and failure modes, such as sensitivity to shortcut features and calibration errors. In this paper, we propose a classifier-free approach for visual counterfactual generation based on Contrastive Analysis (CA). Given two datasets corresponding to different classes (e.g., healthy and patients), we disentangle the generative factors that are common across the two datasets from those that are salient to each dataset, and generate counterfactual images by swapping only the salient factors. By operating directly on data distributions rather than decision boundaries, our method provides model-agnostic VCEs that are less sensitive to classifier biases. Our approach leverages the high-quality synthesis and well-structured latent space of StyleGAN2. We use the feature space F, instead than the usual W-space, to improve detail preservation. Unlike conventional CA approaches, which typically assume salient factors in only one dataset, we introduce an adapted framework and loss functions for VCE that allow multiple salient factors in each dataset. We evaluate our method on three medical imaging datasets and demonstrate superior counterfactual generation quality compared to existing approaches.