# Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation

arXiv: https://arxiv.org/abs/2605.28187 · HTML: https://mlsanity.com/paper/2605.28187/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 49 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 19 of 19)

- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts](https://mlsanity.com/paper/2205.01833/) (2022)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMs](https://mlsanity.com/paper/2305.16339/) (2023)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [Factuality of Large Language Models: A Survey](https://mlsanity.com/paper/2402.02420/) (2024)
- [Predicting Race and Ethnicity From the Sequence of Characters in a Name](https://mlsanity.com/paper/1805.02109/) (2018)
- [AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening](https://mlsanity.com/paper/2504.02870/) (2025)
- [Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance](https://mlsanity.com/paper/2508.19764/) (2025)
- [One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization](https://mlsanity.com/paper/2601.18572/) (2026)
- [No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy](https://mlsanity.com/paper/2509.04404/) (2025)
- [Evaluating the Promise and Pitfalls of LLMs in Hiring Decisions](https://mlsanity.com/paper/2507.02087/) (2025)
- [Avoiding bias when inferring race using name-based approaches](https://mlsanity.com/paper/2104.12553/) (2021)
- [Whose Name Comes Up? Auditing LLM-Based Scholar Recommendations](https://mlsanity.com/paper/2506.00074/) (2025)
- [A Multi-Task Evaluation of LLMs' Processing of Academic Text Input](https://mlsanity.com/paper/2508.11779/) (2025)
- [Who benefits from altmetrics? The effect of team gender composition on the link between online visibility and citation impact](https://mlsanity.com/paper/2308.00405/) (2023)
- [Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendation](https://mlsanity.com/paper/2602.08873/) (2026)
- [Systematic comparison of gender inequality in scientific rankings across disciplines](https://mlsanity.com/paper/2501.13061/) (2025)

## Built on by (most cited first, 1 of 1)

- [Who Gets Named: Citation Type Predicts Individual Naming by Grounded Language Models, and a Roster Instrument Captures 0.5% of It](https://mlsanity.com/paper/2607.23893/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used as scholar recommenders, shaping who is seen as an expert in academia. Existing audits remain English-centric, single discipline, and persona-agnostic, leaving the source of output variability poorly understood. To this end, we propose a benchmark that disentangles the effects of model choice and prompt design on recommendations. We audit 43 LLMs by varying persona prompts (language, location, role-and-task) and context (field, seniority, k). Recommended scholars are compared against Semantic Scholar over six scientific disciplines to measure technical quality (factuality, coverage) and social representativeness (diversity, parity). Basic technical quality is driven by model choice, factuality and parity by context, and diversity by location. South Africa prompts yield less factual lists, while Japan prompts yield highly factual but homogeneous lists skewed toward highly productive scholars. Prompt design is thus a non-trivial axis of LLM-based scholar discovery and should be systematically audited alongside model choice.