# Fairness in Supervised Learning: An Information Theoretic Approach

arXiv: https://arxiv.org/abs/1801.04378 · HTML: https://mlsanity.com/paper/1801.04378/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2018.

## Built on by (most cited first, 17 of 17)

- [InfoFair: Information-Theoretic Intersectional Fairness](https://mlsanity.com/paper/2105.11069/) (2021)
- [Survey on Fair Reinforcement Learning: Theory and Practice](https://mlsanity.com/paper/2205.10032/) (2022)
- [Demystifying Local and Global Fairness Trade-offs in Federated Learning Using Partial Information Decomposition](https://mlsanity.com/paper/2307.11333/) (2023)
- [Gradual (In)Compatibility of Fairness Criteria](https://mlsanity.com/paper/2109.04399/) (2021)
- [Can Information Flows Suggest Targets for Interventions in Neural Circuits?](https://mlsanity.com/paper/2111.05299/) (2021)
- [Exploring Privacy and Fairness Risks in Sharing Diffusion Models: An Adversarial Perspective](https://mlsanity.com/paper/2402.18607/) (2024)
- [Impact of Data Processing on Fairness in Supervised Learning](https://mlsanity.com/paper/2102.01867/) (2021)
- [Information Theoretic Measures for Fairness-aware Feature Selection](https://mlsanity.com/paper/2106.00772/) (2021)
- [Bt-GAN: Generating Fair Synthetic Healthdata via Bias-transforming Generative Adversarial Networks](https://mlsanity.com/paper/2404.13634/) (2024)
- [Fairness-aware Summarization for Justified Decision-Making](https://mlsanity.com/paper/2107.06243/) (2021)
- [Renyi Fair Information Bottleneck for Image Classification](https://mlsanity.com/paper/2203.04950/) (2022)
- [A Unified View of Group Fairness Tradeoffs Using Partial Information Decomposition](https://mlsanity.com/paper/2406.04562/) (2024)
- [Classification Utility, Fairness, and Compactness via Tunable Information Bottleneck and R'enyi Measures](https://mlsanity.com/paper/2206.10043/) (2022)
- [Measuring and Mitigating Bias for Tabular Datasets with Multiple Protected Attributes](https://mlsanity.com/paper/2405.19300/) (2024)
- [Long-Term Fairness in Sequential Multi-Agent Selection with Positive Reinforcement](https://mlsanity.com/paper/2407.07350/) (2024)
- [Does Privacy Always Harm Fairness? Data-Dependent Trade-offs via Chernoff Information Neural Estimation](https://mlsanity.com/paper/2601.13698/) (2026)
- [MIFair: A Mutual-Information Framework for Intersectionality and Multiclass Fairness](https://mlsanity.com/paper/2604.28030/) (2026)

## Abstract (arXiv)

Automated decision making systems are increasingly being used in real-world applications. In these systems for the most part, the decision rules are derived by minimizing the training error on the available historical data. Therefore, if there is a bias related to a sensitive attribute such as gender, race, religion, etc. in the data, say, due to cultural/historical discriminatory practices against a certain demographic, the system could continue discrimination in decisions by including the said bias in its decision rule. We present an information theoretic framework for designing fair predictors from data, which aim to prevent discrimination against a specified sensitive attribute in a supervised learning setting. We use equalized odds as the criterion for discrimination, which demands that the prediction should be independent of the protected attribute conditioned on the actual label. To ensure fairness and generalization simultaneously, we compress the data to an auxiliary variable, which is used for the prediction task. This auxiliary variable is chosen such that it is decontaminated from the discriminatory attribute in the sense of equalized odds. The final predictor is obtained by applying a Bayesian decision rule to the auxiliary variable.