# Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Study

arXiv: https://arxiv.org/abs/2607.06603 · HTML: https://mlsanity.com/paper/2607.06603/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Builds on (most cited first, 29 of 29)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Simple data balancing achieves competitive worst-group-accuracy](https://mlsanity.com/paper/2110.14503/) (2021)
- [FairGAN: Fairness-aware Generative Adversarial Networks](https://mlsanity.com/paper/1805.11202/) (2018)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Adversarially Robust Distillation](https://mlsanity.com/paper/1905.09747/) (2019)
- [Fair Attribute Classification through Latent Space De-biasing](https://mlsanity.com/paper/2012.01469/) (2020)
- [Fair Feature Distillation for Visual Recognition](https://mlsanity.com/paper/2106.04411/) (2021)
- [Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective](https://mlsanity.com/paper/2110.03095/) (2021)
- [Evaluating and Mitigating Bias in Image Classifiers: A Causal Perspective Using Counterfactuals](https://mlsanity.com/paper/2009.08270/) (2020)
- [Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder](https://mlsanity.com/paper/2011.11878/) (2020)
- [SenSeI: Sensitive Set Invariance for Enforcing Individual Fairness](https://mlsanity.com/paper/2006.14168/) (2020)
- [Counterfactual Reasoning for Fair Clinical Risk Prediction](https://mlsanity.com/paper/1907.06260/) (2019)
- [Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluation](https://mlsanity.com/paper/2308.05441/) (2023)
- [Latent Space Smoothing for Individually Fair Representations](https://mlsanity.com/paper/2111.13650/) (2021)
- [Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairness](https://mlsanity.com/paper/2310.19691/) (2023)
- [Counterfactual Fairness Is Basically Demographic Parity](https://mlsanity.com/paper/2208.03843/) (2022)
- [Improving Fairness using Vision-Language Driven Image Augmentation](https://mlsanity.com/paper/2311.01573/) (2023)
- [Consistency and Accuracy of CelebA Attribute Values](https://mlsanity.com/paper/2210.07356/) (2022)
- [When causation does not imply correlation: robust violations of the Faithfulness axiom](https://mlsanity.com/paper/1505.03118/) (2015)

## Built on by (most cited first, 1 of 1)

- [Causally Fair Generation with Large Language Models](https://mlsanity.com/paper/2610.04444/) (2026)

## Abstract (arXiv)

The notion of algorithmic fairness has been actively explored from various aspects of fairness, such as counterfactual fairness (CF) and group fairness (GF). However, the exact relationship between CF and GF remains to be unclear, especially in image classification tasks; the reason is because we often cannot collect counterfactual samples regarding a sensitive attribute, essential for evaluating CF, from the existing images (eg, a photo of the same person but with different secondary sex characteristics). In this paper, we construct new image datasets for evaluating CF by using a high-quality image editing method and carefully labeling with human annotators. Our datasets, oursceleb and ourslfw, build upon the popular image GF benchmarks; hence, we can evaluate CF and GF simultaneously. We empirically observe that CF does not imply GF in image classification, whereas previous studies on tabular datasets observed the opposite. We theoretically show that it could be due to the existence of a latent attribute G that is correlated with, but not caused by, the sensitive attribute (eg, secondary sex characteristics are highly correlated with hair length). From this observation, we propose a simple baseline, Counterfactual Knowledge Distillation (CKD), to mitigate such correlation with the sensitive attributes. Extensive experimental results on oursceleb and ourslfw demonstrate that CF-achieving models satisfy GF if we successfully reduce the reliance on G (eg, using CKD).