# Fairness Perceptions of Algorithmic Decision-Making: A Systematic Review of the Empirical Literature

arXiv: https://arxiv.org/abs/2103.12016 · HTML: https://mlsanity.com/paper/2103.12016/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 132 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 25 of 25)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Fairness in Machine Learning: Lessons from Political Philosophy](https://mlsanity.com/paper/1712.03586/) (2017)
- [Artificial Intelligence: the global landscape of ethics guidelines](https://mlsanity.com/paper/1906.11668/) (2019)
- [Re-imagining Algorithmic Fairness in India and Beyond](https://mlsanity.com/paper/2101.09995/) (2021)
- [50 Years of Test (Un)fairness: Lessons for Machine Learning](https://mlsanity.com/paper/1811.10104/) (2018)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [On Formalizing Fairness in Prediction with Machine Learning](https://mlsanity.com/paper/1710.03184/) (2017)
- [From Parity to Preference-based Notions of Fairness in Classification](https://mlsanity.com/paper/1707.00010/) (2017)
- [Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning](https://mlsanity.com/paper/1902.04783/) (2019)
- [How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness](https://mlsanity.com/paper/1811.03654/) (2018)
- [Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences](https://mlsanity.com/paper/2001.09604/) (2020)
- [Society-in-the-Loop: Programming the Algorithmic Social Contract](https://mlsanity.com/paper/1707.07232/) (2017)
- [Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction](https://mlsanity.com/paper/1802.09548/) (2018)
- [Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment](https://mlsanity.com/paper/1901.07694/) (2019)
- [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021)
- [Dimensions of Diversity in Human Perceptions of Algorithmic Fairness](https://mlsanity.com/paper/2005.00808/) (2020)
- [A Study on Fairness and Trust Perceptions in Automated Decision Making](https://mlsanity.com/paper/2103.04757/) (2021)
- [Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics](https://mlsanity.com/paper/2001.00089/) (2019)
- [The Threats of Artificial Intelligence Scale (TAI). Development, Measurement and Test Over Three Application Domains](https://mlsanity.com/paper/2006.07211/) (2020)
- [Exploring User Opinions of Fairness in Recommender Systems](https://mlsanity.com/paper/2003.06461/) (2020)
- [The FairCeptron: A Framework for Measuring Human Perceptions of Algorithmic Fairness](https://mlsanity.com/paper/2102.04119/) (2021)
- [Artificial Intelligence as an Anti-Corruption Tool (AI-ACT) -- Potentials and Pitfalls for Top-down and Bottom-up Approaches](https://mlsanity.com/paper/2102.11567/) (2021)

## Built on by (most cited first, 27 of 27)

- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI](https://mlsanity.com/paper/2308.07213/) (2023)
- [ABCFair: an Adaptable Benchmark approach for Comparing Fairness Methods](https://mlsanity.com/paper/2409.16965/) (2024)
- [Local Causal Discovery for Structural Evidence of Direct Discrimination](https://mlsanity.com/paper/2405.14848/) (2024)
- [On the Relationship Between Explanations, Fairness Perceptions, and Decisions](https://mlsanity.com/paper/2204.13156/) (2022)
- [Artificial intelligence in government: Concepts, standards, and a unified framework](https://mlsanity.com/paper/2210.17218/) (2022)
- [Personalized Help for Optimizing Low-Skilled Users' Strategy](https://mlsanity.com/paper/2411.09109/) (2024)
- [Fair Enough? A map of the current limitations of the requirements to have fair algorithms](https://mlsanity.com/paper/2311.12435/) (2023)
- [Converging Measures and an Emergent Model: A Meta-Analysis of Human-Automation Trust Questionnaires](https://mlsanity.com/paper/2303.13799/) (2023)
- [Trustworthy, responsible, ethical AI in manufacturing and supply chains: synthesis and emerging research questions](https://mlsanity.com/paper/2305.11581/) (2023)
- [Fairness Certification for Natural Language Processing and Large Language Models](https://mlsanity.com/paper/2401.01262/) (2024)
- [Legally Binding but Unfair? Towards Assessing Fairness of Privacy Policies](https://mlsanity.com/paper/2403.08115/) (2024)
- [Mapping the Potential of Explainable AI for Fairness Along the AI Lifecycle](https://mlsanity.com/paper/2404.18736/) (2024)
- [Challenging the Human-in-the-loop in Algorithmic Decision-making](https://mlsanity.com/paper/2405.10706/) (2024)
- [From Expectation to Habit: Why Do Software Practitioners Adopt Fairness Toolkits?](https://mlsanity.com/paper/2412.13846/) (2024)
- [A Multi-Objective Evaluation Framework for Analyzing Utility-Fairness Trade-Offs in Machine Learning Systems](https://mlsanity.com/paper/2503.11120/) (2025)
- [Tracing Stereotypes in Pre-trained Transformers: From Biased Neurons to Fairer Models](https://mlsanity.com/paper/2601.05663/) (2026)
- ["Is This Not Enough?": Asymmetries in Institutional Accountability and Collective Sensemaking in the Case of Canada's Algorithmic Visa Triage System](https://mlsanity.com/paper/2606.13071/) (2026)
- [Connecting Algorithmic Fairness to Quality Dimensions in Machine Learning in Official Statistics and Survey Production](https://mlsanity.com/paper/2402.09328/) (2024)
- [Perceptions of Discriminatory Decisions of Artificial Intelligence: Unpacking the Role of Individual Characteristics](https://mlsanity.com/paper/2410.13250/) (2024)
- [Revisiting Technical Bias Mitigation Strategies](https://mlsanity.com/paper/2410.17433/) (2024)
- [Data Preparation for Fairness-Performance Trade-Offs: A Practitioner-Friendly Alternative?](https://mlsanity.com/paper/2412.15920/) (2024)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Where are the Humans? A Scoping Review of Fairness in Multi-agent AI Systems](https://mlsanity.com/paper/2604.15078/) (2026)
- [From Awareness to Action: Understanding and Overcoming the Research-Practice Gap in Algorithmic Fairness for Public Health](https://mlsanity.com/paper/2606.11214/) (2026)
- [Can LLMs Rank? A Tale of Triads and Triage](https://mlsanity.com/paper/2606.30412/) (2026)

## Abstract (arXiv)

Algorithmic decision-making (ADM) increasingly shapes people's daily lives. Given that such autonomous systems can cause severe harm to individuals and social groups, fairness concerns have arisen. A human-centric approach demanded by scholars and policymakers requires taking people's fairness perceptions into account when designing and implementing ADM. We provide a comprehensive, systematic literature review synthesizing the existing empirical insights on perceptions of algorithmic fairness from 39 empirical studies spanning multiple domains and scientific disciplines. Through thorough coding, we systemize the current empirical literature along four dimensions: (a) algorithmic predictors, (b) human predictors, (c) comparative effects (human decision-making vs. algorithmic decision-making), and (d) consequences of ADM. While we identify much heterogeneity around the theoretical concepts and empirical measurements of algorithmic fairness, the insights come almost exclusively from Western-democratic contexts. By advocating for more interdisciplinary research adopting a society-in-the-loop framework, we hope our work will contribute to fairer and more responsible ADM.