# What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring

arXiv: https://arxiv.org/abs/1909.03567 · HTML: https://mlsanity.com/paper/1909.03567/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2019.

## Built on by (most cited first, 14 of 14)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models](https://mlsanity.com/paper/2309.08902/) (2023)
- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023)
- [Investigations of Performance and Bias in Human-AI Teamwork in Hiring](https://mlsanity.com/paper/2202.11812/) (2022)
- [Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance](https://mlsanity.com/paper/2207.01482/) (2022)
- [Social Norm Bias: Residual Harms of Fairness-Aware Algorithms](https://mlsanity.com/paper/2108.11056/) (2021)
- [On the Relationship Between Explanations, Fairness Perceptions, and Decisions](https://mlsanity.com/paper/2204.13156/) (2022)
- [From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions](https://mlsanity.com/paper/2406.18865/) (2024)
- [Gender Bias in Transformer Models: A comprehensive survey](https://mlsanity.com/paper/2306.10530/) (2023)
- [Algorithmic Fairness: A Tolerance Perspective](https://mlsanity.com/paper/2405.09543/) (2024)
- [Algorithmic Hiring and Diversity: Reducing Human-Algorithm Similarity for Better Outcomes](https://mlsanity.com/paper/2505.14388/) (2025)
- [NLPGuard: A Framework for Mitigating the Use of Protected Attributes by NLP Classifiers](https://mlsanity.com/paper/2407.01697/) (2024)

## Abstract (arXiv)

Although systematic biases in decision-making are widely documented, the ways in which they emerge from different sources is less understood. We present a controlled experimental platform to study gender bias in hiring by decoupling the effect of world distribution (the gender breakdown of candidates in a specific profession) from bias in human decision-making. We explore the effectiveness of textitrepresentation criteria, fixed proportional display of candidates, as an intervention strategy for mitigation of gender bias by conducting experiments measuring human decision-makers' rankings for who they would recommend as potential hires. Experiments across professions with varying gender proportions show that balancing gender representation in candidate slates can correct biases for some professions where the world distribution is skewed, although doing so has no impact on other professions where human persistent preferences are at play. We show that the gender of the decision-maker, complexity of the decision-making task and over- and under-representation of genders in the candidate slate can all impact the final decision. By decoupling sources of bias, we can better isolate strategies for bias mitigation in human-in-the-loop systems.