mlsanityData to 7 Oct 2026

MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions

arXiv 2606.16562 ↗First posted 15 Jun 2026cs.LGcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Five years after the discovery of persistent anti-Muslim bias in large language models, most evaluations remain confined to single-turn prompt completion, a setting that no longer reflects how frontier LLMs are deployed. We introduce MIRAGE (Muslim-Identity Reasoning and Agentic Generation Evaluation), a benchmark of 1,200 prompts spanning three deployment-realistic conditions: direct completion, chain-of-thought reasoning, and simulated agentic decision-making across content moderation, lending triage, refugee claim summarization, and hiring screens. Across six frontier models, we find that (i) chain-of-thought reasoning amplifies rather than suppresses Muslim-violence associations by 12--34% relative to direct completion, (ii) agentic decisions exhibit a 9--22 percentage-point asymmetry between Muslim and matched non-Muslim cases on identical evidence, and (iii) bias is sharply time-coupled to retrieved news context, increasing 18--27% under recent-conflict retrieval. Existing prompt-based mitigations transfer poorly across our three conditions, suppressing direct-completion bias while leaving agentic asymmetry largely intact. We release MIRAGE and an open evaluation harness to support targeted mitigation research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 20 of its 31 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

20 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic20 Feb

Other references · 19

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  6. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  10. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  11. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  12. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
7 more
  1. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  2. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  3. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 77
  4. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  5. 2024Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attributioncited by 39
  6. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  7. 2025Prompt Engineering Techniques for Mitigating Cultural Bias Against Arabs and Muslims in Large Language Models: A Systematic Reviewcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026IslamicLegalBench: Evaluating LLMs Knowledge and Reasoning of Islamic Law Across 1,200 Years of Islamic Pluralist Legal Traditions
  2. 2025Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMs
  3. 2025Debiasing LLMs by Masking Unfairness-Driving Attention Heads
  4. 2026BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs
  5. 2025Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias
  6. 2025Evaluating Large Language Models for Detecting Antisemitism
  7. 2025Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge
  8. 2024A Multi-LLM Debiasing Framework