# Mutatis Mutandis: Revisiting the Comparator in Discrimination Testing

arXiv: https://arxiv.org/abs/2405.13693 · HTML: https://mlsanity.com/paper/2405.13693/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 114 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 30 of 32)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [D'ya like DAGs? A Survey on Structure Learning and Causal Discovery](https://mlsanity.com/paper/2103.02582/) (2021)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Controllable Invariance through Adversarial Feature Learning](https://mlsanity.com/paper/1705.11122/) (2017)
- [On the Apparent Conflict Between Individual and Group Fairness](https://mlsanity.com/paper/1912.06883/) (2019)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Invariant Representations without Adversarial Training](https://mlsanity.com/paper/1805.09458/) (2018)
- [50 Years of Test (Un)fairness: Lessons for Machine Learning](https://mlsanity.com/paper/1811.10104/) (2018)
- [Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI](https://mlsanity.com/paper/2005.05906/) (2020)
- [Fairness Testing: A Comprehensive Survey and Analysis of Trends](https://mlsanity.com/paper/2207.10223/) (2022)
- [Metric Learning for Individual Fairness](https://mlsanity.com/paper/1906.00250/) (2019)
- [Discrimination in the Age of Algorithms](https://mlsanity.com/paper/1902.03731/) (2019)
- [FlipTest: Fairness Testing via Optimal Transport](https://mlsanity.com/paper/1906.09218/) (2019)
- [Causal normalizing flows: from theory to practice](https://mlsanity.com/paper/2306.05415/) (2023)
- [Empirical observation of negligible fairness-accuracy trade-offs in machine learning for public policy](https://mlsanity.com/paper/2012.02972/) (2020)
- [What's Sex Got To Do With Fair Machine Learning?](https://mlsanity.com/paper/2006.01770/) (2020)
- [Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairness](https://mlsanity.com/paper/2310.19691/) (2023)
- [Fairness Under Feature Exemptions: Counterfactual and Observational Measures](https://mlsanity.com/paper/2006.07986/) (2020)
- [Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree](https://mlsanity.com/paper/2305.13938/) (2023)
- [Causal Inference for Social Discrimination Reasoning](https://mlsanity.com/paper/1608.03735/) (2016)
- [Counterfactual Situation Testing: Uncovering Discrimination under Fairness given the Difference](https://mlsanity.com/paper/2302.11944/) (2023)
- [What is Proxy Discrimination?](https://mlsanity.com/paper/2205.05265/) (2022)
- [Fair Group-Shared Representations with Normalizing Flows](https://mlsanity.com/paper/2201.06336/) (2022)

## Built on by (most cited first, 2 of 2)

- [Counterfactual Situation Testing: From Single to Multidimensional Discrimination](https://mlsanity.com/paper/2502.01267/) (2025)
- [Implementing Causal Perception: Competing SCMs and Situated Fairness](https://mlsanity.com/paper/2608.03917/) (2026)

## Abstract (arXiv)

Testing for individual discrimination involves deriving a profile, the comparator, similar to the one making the discrimination claim, the complainant, based on a protected attribute, such as race or gender, and comparing their decision outcomes. The complainant-comparator pair is central to discrimination testing. Most discrimination testing tools rely on this pair to establish evidence for discrimination. In this work, we revisit the role of the comparator in discrimination testing. We first argue for the inherent causal modeling nature of deriving the comparator. We then introduce a two-kind classification for the comparator: the ceteris paribus, or "with all else equal," (CP) comparator and the mutatis mutandis, or "with the appropriate adjustments being made," (MM) comparator. The CP comparator is the standard comparator, representing an idealized comparison for establishing discrimination as it aims for a complainant-comparator pair that only differs in membership in the protected attribute. As an alternative to the CP comparator, we define the MM comparator, which requires a comparator that represents the ``what would have been'' of the complainant without the effects of the protected attribute on the non-protected attributes. Under the MM comparator, the complainant-comparator pair can be dissimilar in terms of the non-protected attributes, departing from the idealized comparison imposed by the CP comparator. Notably, the MM comparator denotes a more complex object and its implementation offers an impactful venue for machine learning methods. We illustrate these two comparators and their impact on discrimination testing using a real-world example.