# Testing for Causal Fairness

arXiv: https://arxiv.org/abs/2502.12874 · HTML: https://mlsanity.com/paper/2502.12874/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 30 of 31)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Survey on Causal Inference](https://mlsanity.com/paper/2002.02770/) (2020)
- [Are Large Pre-Trained Language Models Leaking Your Personal Information?](https://mlsanity.com/paper/2205.12628/) (2022)
- [FairFed: Enabling Group Fairness in Federated Learning](https://mlsanity.com/paper/2110.00857/) (2021)
- [Causal Machine Learning: A Survey and Open Problems](https://mlsanity.com/paper/2206.15475/) (2022)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [Causal Fairness Analysis](https://mlsanity.com/paper/2207.11385/) (2022)
- [A Wild Bootstrap for Degenerate Kernel Tests](https://mlsanity.com/paper/1408.5404/) (2014)
- [Privacy and Fairness in Federated Learning: on the Perspective of Trade-off](https://mlsanity.com/paper/2306.14123/) (2023)
- [A Statistical Test for Probabilistic Fairness](https://mlsanity.com/paper/2012.04800/) (2020)
- [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://mlsanity.com/paper/2310.04993/) (2023)
- [Responsible AI: Gender bias assessment in emotion recognition](https://mlsanity.com/paper/2103.11436/) (2021)
- [Proximal Learning for Individualized Treatment Regimes Under Unmeasured Confounding](https://mlsanity.com/paper/2105.01187/) (2021)
- [On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models](https://mlsanity.com/paper/2403.04204/) (2024)
- [Counterfactual Fairness with Partially Known Causal Graph](https://mlsanity.com/paper/2205.13972/) (2022)
- [Integer Programming for Causal Structure Learning in the Presence of Latent Variables](https://mlsanity.com/paper/2102.03129/) (2021)
- [Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model](https://mlsanity.com/paper/2306.01424/) (2023)
- [Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach](https://mlsanity.com/paper/2303.03300/) (2023)
- [Counterfactual Density Estimation using Kernel Stein Discrepancies](https://mlsanity.com/paper/2309.16129/) (2023)
- [Interventional Fairness on Partially Known Causal Graphs: A Constrained Optimization Approach](https://mlsanity.com/paper/2401.10632/) (2024)
- [Debiased Collaborative Filtering with Kernel-Based Causal Balancing](https://mlsanity.com/paper/2404.19596/) (2024)
- [Examining Multimodal Gender and Content Bias in ChatGPT-4o](https://mlsanity.com/paper/2411.19140/) (2024)
- [Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems](https://mlsanity.com/paper/2405.20626/) (2024)

## Abstract (arXiv)

Causality is widely used in fairness analysis to prevent discrimination on sensitive attributes, such as genders in career recruitment and races in crime prediction. However, the current data-based Potential Outcomes Framework (POF) often leads to untrustworthy fairness analysis results when handling high-dimensional data. To address this, we introduce a distribution-based POF that transform fairness analysis into Distributional Closeness Testing (DCT) by intervening on sensitive attributes. We define counterfactual closeness fairness as the null hypothesis of DCT, where a sensitive attribute is considered fair if its factual and counterfactual potential outcome distributions are sufficiently close. We introduce the Norm-Adaptive Maximum Mean Discrepancy Treatment Effect (N-TE) as a statistic for measuring distributional closeness and apply DCT using the empirical estimator of NTE, referred to Counterfactual Fairness-CLOseness Testing (textrmCF-CLOT). To ensure the trustworthiness of testing results, we establish the testing consistency of N-TE through rigorous theoretical analysis. textrmCF-CLOT demonstrates sensitivity in fairness analysis through the flexibility of the closeness parameter ε. Unfair sensitive attributes have been successfully tested by textrmCF-CLOT in extensive experiments across various real-world scenarios, which validate the consistency of the testing.