Prestige over merit: An adapted audit of LLM bias in peer review
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) play a growing but largely informal role in scholarly peer review. Yet whether LLMs reproduce biases observed in human decision-making remains unclear. We adapt a resume-style audit to scientific publishing, developing a multi-role LLM simulation (editor/reviewer) that evaluates high-quality manuscripts across the physical, biological, and social sciences under randomized author identities (institutional prestige, gender, race). Revealing author identities lowers reviewer rejection recommendations by roughly 25% of the mean rejection rate despite identical content, indicating that status cues beyond paper quality shape outcomes. Institutional prestige is the dominant cue: papers attributed to low-prestige affiliations receive lower quality scores in every field, a penalty that survives family-wise multiple-testing correction at the editor stage. Effects at the rejection margin are smaller and mostly fragile to correction, with one robust intersectional exception. Relative to male authors, female authors at low-prestige institutions receive lower reviewer quality scores and more rejection recommendations than those at high-prestige institutions. To probe mechanisms, we generate synthetic CVs for the same author profiles; these encode large prestige-linked disparities and an inverted prestige-tenure gradient relative to national benchmarks. The results suggest that domain norms and prestige-linked priors embedded in training data shape outcomes once identity is visible.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Sep 2025.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Paraphrasing Adversarial Attack on LLM-as-a-Reviewercited by 8
- 2026Need or Reputation? Region, National Income, and Academic Status in LLM Gatekeeping Decisionscited by 0
- 2026Institutional Prestige as Geographic Bias in Large Language Models: Evidence from Three Factorial Experiments with Bootstrap Confidence Intervalscited by 0
- 2026Difference-in-Differences on a Censored Rating Scale Can Manufacture an Effect: Evidence from a Pre-Registered LLM-Judge Auditcited by 0
- 2026Authority Bias in Conversational Search Engines for Academic Paper Recommendationcited by 0
- 2026Authority Bias in Conversational Search Engines for Academic Paper Recommendation31 Aug 2026
- 2026Difference-in-Differences on a Censored Rating Scale Can Manufacture an Effect: Evidence from a Pre-Registered LLM-Judge Audit27 Aug 2026
- 2026Need or Reputation? Region, National Income, and Academic Status in LLM Gatekeeping Decisions27 Jun 2026
- 2026Institutional Prestige as Geographic Bias in Large Language Models: Evidence from Three Factorial Experiments with Bootstrap Confidence Intervals10 Jun 2026
- 2026Paraphrasing Adversarial Attack on LLM-as-a-Reviewer11 Jan 2026