# Fair Inputs and Fair Outputs: The Incompatibility of Fairness in Privacy and Accuracy

arXiv: https://arxiv.org/abs/2005.09209 · HTML: https://mlsanity.com/paper/2005.09209/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2020.

## Built on by (most cited first, 6 of 6)

- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [Dikaios: Privacy Auditing of Algorithmic Fairness via Attribute Inference Attacks](https://mlsanity.com/paper/2202.02242/) (2022)
- [From Principle to Practice: Vertical Data Minimization for Machine Learning](https://mlsanity.com/paper/2311.10500/) (2023)
- [Beyond traditional assumptions in fair machine learning](https://mlsanity.com/paper/2101.12476/) (2021)
- [I Prefer not to Say: Protecting User Consent in Models with Optional Personal Data](https://mlsanity.com/paper/2210.13954/) (2022)
- [SoK: Data Minimization in Machine Learning](https://mlsanity.com/paper/2508.10836/) (2025)

## Abstract (arXiv)

Fairness concerns about algorithmic decision-making systems have been mainly focused on the outputs (e.g., the accuracy of a classifier across individuals or groups). However, one may additionally be concerned with fairness in the inputs. In this paper, we propose and formulate two properties regarding the inputs of (features used by) a classifier. In particular, we claim that fair privacy (whether individuals are all asked to reveal the same information) and need-to-know (whether users are only asked for the minimal information required for the task at hand) are desirable properties of a decision system. We explore the interaction between these properties and fairness in the outputs (fair prediction accuracy). We show that for an optimal classifier these three properties are in general incompatible, and we explain what common properties of data make them incompatible. Finally we provide an algorithm to verify if the trade-off between the three properties exists in a given dataset, and use the algorithm to show that this trade-off is common in real data.