# A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness

arXiv: https://arxiv.org/abs/1911.03020 · HTML: https://mlsanity.com/paper/1911.03020/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 11 of 11)

- [Perspectives on Incorporating Expert Feedback into Model Updates](https://mlsanity.com/paper/2205.06905/) (2022)
- [InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions](https://mlsanity.com/paper/2210.07440/) (2022)
- [Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways](https://mlsanity.com/paper/2402.04420/) (2024)
- [Toward A Causal Framework for Modeling Perception](https://mlsanity.com/paper/2401.13408/) (2024)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Mapping the Potential of Explainable AI for Fairness Along the AI Lifecycle](https://mlsanity.com/paper/2404.18736/) (2024)
- [Fair Clustering: Critique, Caveats, and Future Directions](https://mlsanity.com/paper/2406.15960/) (2024)
- [Understanding Decision Subjects' Engagement with and Perceived Fairness of AI Models When Opportunities of Qualification Improvement Exist](https://mlsanity.com/paper/2410.03126/) (2024)
- [On Learning and Enforcing Latent Assessment Models using Binary Feedback from Human Auditors Regarding Black-Box Classifiers](https://mlsanity.com/paper/2202.08250/) (2022)
- [Achievement and Fragility of Long-term Equitability](https://mlsanity.com/paper/2206.12333/) (2022)
- [Implementing Causal Perception: Competing SCMs and Situated Fairness](https://mlsanity.com/paper/2608.03917/) (2026)

## Abstract (arXiv)

Existing mathematical notions of fairness fail to account for the context of decision-making. We argue that moral consideration of contextual factors is an inherently human task. So we present a framework to learn context-aware mathematical formulations of fairness by eliciting people's situated fairness assessments. Our family of fairness notions corresponds to a new interpretation of economic models of Equality of Opportunity (EOP), and it includes most existing notions of fairness as special cases. Our human-in-the-loop approach is designed to learn the appropriate parameters of the EOP family by utilizing human responses to pair-wise questions about decision subjects' circumstance and deservingness, and the harm/benefit imposed on them. We illustrate our framework in a hypothetical criminal risk assessment scenario by conducting a series of human-subject experiments on Amazon Mechanical Turk. Our work takes an important initial step toward empowering stakeholders to have a voice in the formulation of fairness for Machine Learning.