Conditional Cognitive Biases in LLMs: How Biased User Turns Modulate In-Context Reasoning
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
We present an evaluation of cognitive bias expression in state-of-the-art instruction-tuned LLMs under realistic multi-turn interaction settings. Our work introduces a novel three-condition experimental framework that disentangles the effect of exposure to a biased user turn from the effect of the turn's semantic content, alongside a benchmark of 24,300 jury-validated user prompts spanning all 81 cells of a 9x9 target-human bias interaction matrix. Across eight frontier LLMs, we find that biased conversational context systematically increases bias expression relative to zero-shot baselines in 6 of 8 models. We identify two competing behavioral dynamics underlying this effect: conversational exposure to biased reasoning generally amplifies downstream bias tendencies, while explicitly stated bias cues often trigger alignment-related suppression behaviors that reduce overt bias expression. We release our framework, codebase, and dataset to support future research on context-conditioned cognitive biases and behavioral adaptation in LLMs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 39 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
22 of 39 references.
Closest ancestors · 2
Same direction, newest first
- 2022Thinking Fast and Slow in Large Language Models10 Dec
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?25 Feb
Other references · 20
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
- 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
8 more
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2023Whose Opinions Do Language Models Reflect?cited by 773
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2022A Survey on In-context Learningcited by 721
- 2022Using cognitive psychology to understand GPT-3cited by 398
- 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 98
- 2024A Comprehensive Evaluation of Cognitive Biases in LLMscited by 34
- 2026Prototype Guided Post-pretraining for Single-Cell Representation Learningcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
- 2026BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs
- 2024CBEval: A framework for evaluating and interpreting cognitive biases in LLMs
- 2026Mechanics of Bias and Reasoning: Interpreting the Impact of Chain-of-Thought Prompting on Gender Bias in LLMs
- 2026How Implicit Bias Accumulates and Propagates in LLM Long-term Memory
- 2026Inference-Time Reasoning Selectively Reduces Implicit Social Bias in Large Language Models
- 2026Emulating Aggregate Human Choice Behavior and Biases with GPT Conversational Agents
- 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias