When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance
Not on the map. Why?
Abstract (arXiv)
We ask whether large language models (LLMs) treat queries about religious conversion symmetrically. The answer is no. When asked for advice on hypothetical faith transitions from religion A->B vs. religion B->A , models exhibited consistent asymmetries, favoring some religions while subtly discouraging conversion to others. On average Catholic, Bah'a''i, and Sikh religions were broadly favored (high support for joining, low support for leaving), while Atheists, Agnostics, and Jehovah's Witnesses were primarily disfavored. Patterns varied by model size and model provider, with Grok 4.20 exhibiting the strongest asymmetries. We tested 20 commercial and open-source language models across 182 religion pairings using a human-verified LLM-as-judge framework. Each model was probed via interactions with a simulated user asking for advice on a potential faith conversion. Models tended to use more encouraging language for some faith transitions over others; these patterns were systematically repeatable across multiple trials. All LLMs tested exhibited reproducible asymmetry, though the pattern of preferences differed for each. Overall preferences persist across multiple question phrasings and variations in the religious pairing dataset. Taken together, these results suggest that asymmetry is a robust property of model behavior rather than an artifact of how the models' answers were scored. It is important to consider that any imbalances deployed and reproduced at scale can have real-world implications.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 52 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
18 of 52 references.
References · 18
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
- 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
- 2024What Large Language Models Know and What People Think They Knowcited by 109
- 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 99
- 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
- 2023Knowledge of cultural moral norms in large language modelscited by 51
6 more
- 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
- 2023The Persuasive Power of Large Language Modelscited by 46
- 2024Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLMcited by 16
- 2025How Deep Is Representational Bias in LLMs? The Cases of Caste and Religioncited by 14
- 2024Divine LLaMAs: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Modelscited by 12
- 2022Debiased Large Language Models Still Associate Muslims with Uniquely Violent Actscited by 8
Built on by
Citing papers per year: 2026: 1.
1 paper.