# AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment

arXiv: https://arxiv.org/abs/2409.16022 · HTML: https://mlsanity.com/paper/2409.16022/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 70 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Pre-Trained Language Models for Interactive Decision-Making](https://mlsanity.com/paper/2202.01771/) (2022)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [Measuring Political Bias in Large Language Models: What Is Said and How It Is Said](https://mlsanity.com/paper/2403.18932/) (2024)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [Supporting Human-AI Collaboration in Auditing LLMs with LLMs](https://mlsanity.com/paper/2304.09991/) (2023)
- [UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor](https://mlsanity.com/paper/2406.06519/) (2024)
- [Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior](https://mlsanity.com/paper/2107.09234/) (2021)
- [Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media](https://mlsanity.com/paper/1704.01347/) (2017)
- [Task Supportive and Personalized Human-Large Language Model Interaction: A User Study](https://mlsanity.com/paper/2402.06170/) (2024)
- [Cognitively Biased Users Interacting with Algorithmically Biased Results in Whole-Session Search on Debated Topics](https://mlsanity.com/paper/2403.17286/) (2024)
- [Decoy Effect In Search Interaction: Understanding User Behavior and Measuring System Vulnerability](https://mlsanity.com/paper/2403.18462/) (2024)

## Built on by (most cited first, 20 of 20)

- [Anchoring Bias in Large Language Models: An Experimental Study](https://mlsanity.com/paper/2412.06593/) (2024)
- [LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions](https://mlsanity.com/paper/2508.18321/) (2025)
- [Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluation](https://mlsanity.com/paper/2503.19092/) (2025)
- [Herd Behavior: Investigating Peer Influence in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2505.21588/) (2025)
- [Mitigating Selection Bias with Node Pruning and Auxiliary Options](https://mlsanity.com/paper/2409.18857/) (2024)
- [Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations](https://mlsanity.com/paper/2502.01349/) (2025)
- [The Bias is in the Details: An Assessment of Cognitive Bias in LLMs](https://mlsanity.com/paper/2509.22856/) (2025)
- [Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models](https://mlsanity.com/paper/2604.02485/) (2026)
- [Self-Adaptive Cognitive Debiasing for Large Language Models in Decision-Making](https://mlsanity.com/paper/2504.04141/) (2025)
- [Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Search](https://mlsanity.com/paper/2504.02074/) (2025)
- [Anchors in the Machine: Behavioral and Attributional Evidence of Anchoring Bias in LLMs](https://mlsanity.com/paper/2511.05766/) (2025)
- [The Decoy Dilemma in Online Medical Information Evaluation: A Comparative Study of Credibility Assessments by LLM and Human Judges](https://mlsanity.com/paper/2411.15396/) (2024)
- [Mitigating the Threshold Priming Effect in Large Language Model-Based Relevance Judgments via Personality Infusing](https://mlsanity.com/paper/2512.00390/) (2025)
- [Judging with Personality and Confidence: A Study on Personality-Conditioned LLM Relevance Assessment](https://mlsanity.com/paper/2601.01862/) (2026)
- [AMEL: Accumulated Message Effects on LLM Judgments](https://mlsanity.com/paper/2605.22714/) (2026)
- [Persona Conditioning as an Assessor-Sensitivity Probe for LLM-Based IR Evaluation](https://mlsanity.com/paper/2608.10385/) (2026)
- [Re-Rankers as Relevance Judges](https://mlsanity.com/paper/2601.04455/) (2026)
- [Stability Regimes for Framing-Sensitive Fine-Tuning in Language Models](https://mlsanity.com/paper/2602.15273/) (2026)
- [The Effects of Visual Priming on Cooperative Behavior in Vision-Language Models](https://mlsanity.com/paper/2604.27953/) (2026)
- [Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection](https://mlsanity.com/paper/2606.30587/) (2026)

## Abstract (arXiv)

Cognitive biases are systematic deviations in thinking that lead to irrational judgments and problematic decision-making, extensively studied across various fields. Recently, large language models (LLMs) have shown advanced understanding capabilities but may inherit human biases from their training data. While social biases in LLMs have been well-studied, cognitive biases have received less attention, with existing research focusing on specific scenarios. The broader impact of cognitive biases on LLMs in various decision-making contexts remains underexplored. We investigated whether LLMs are influenced by the threshold priming effect in relevance judgments, a core task and widely-discussed research topic in the Information Retrieval (IR) coummunity. The priming effect occurs when exposure to certain stimuli unconsciously affects subsequent behavior and decisions. Our experiment employed 10 topics from the TREC 2019 Deep Learning passage track collection, and tested AI judgments under different document relevance scores, batch lengths, and LLM models, including GPT-3.5, GPT-4, LLaMa2-13B and LLaMa2-70B. Results showed that LLMs tend to give lower scores to later documents if earlier ones have high relevance, and vice versa, regardless of the combination and model used. Our finding demonstrates that LLM%u2019s judgments, similar to human judgments, are also influenced by threshold priming biases, and suggests that researchers and system engineers should take into account potential human-like cognitive biases in designing, evaluating, and auditing LLMs in IR tasks and beyond.