# Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs

arXiv: https://arxiv.org/abs/2507.07186 · HTML: https://mlsanity.com/paper/2507.07186/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 9 Jul 2025.

## Builds on (most cited first, 30 of 33)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [Benchmarking Cognitive Biases in Large Language Models as Evaluators](https://mlsanity.com/paper/2309.17012/) (2023)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [LoRA vs Full Fine-tuning: An Illusion of Equivalence](https://mlsanity.com/paper/2410.21228/) (2024)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data](https://mlsanity.com/paper/2407.14985/) (2024)
- [Fairness in Large Language Models: A Taxonomic Survey](https://mlsanity.com/paper/2404.01349/) (2024)
- [The Impact of Initialization on LoRA Finetuning Dynamics](https://mlsanity.com/paper/2406.08447/) (2024)
- [Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias](https://mlsanity.com/paper/2308.00225/) (2023)
- [Addressing cognitive bias in medical language models](https://mlsanity.com/paper/2402.08113/) (2024)
- [An Empirical Comparison of Instance Attribution Methods for NLP](https://mlsanity.com/paper/2104.04128/) (2021)
- [Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting](https://mlsanity.com/paper/2110.05367/) (2021)
- [A Comprehensive Evaluation of Cognitive Biases in LLMs](https://mlsanity.com/paper/2410.15413/) (2024)
- [Enhancing Diagnostic Accuracy through Multi-Agent Conversations: Using Large Language Models to Mitigate Cognitive Bias](https://mlsanity.com/paper/2401.14589/) (2024)
- [Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis](https://mlsanity.com/paper/2502.11812/) (2025)
- [AGR: Age Group fairness Reward for Bias Mitigation in LLMs](https://mlsanity.com/paper/2409.04340/) (2024)
- [CBEval: A framework for evaluating and interpreting cognitive biases in LLMs](https://mlsanity.com/paper/2412.03605/) (2024)
- [Language models are susceptible to incorrect patient self-diagnosis in medical applications](https://mlsanity.com/paper/2309.09362/) (2023)
- [Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation](https://mlsanity.com/paper/2309.12491/) (2023)
- [Comparing the Framing Effect in Humans and LLMs on Naturally Occurring Texts](https://mlsanity.com/paper/2502.17091/) (2025)
- [Comparing Rationality Between Large Language Models and Humans: Insights and Open Questions](https://mlsanity.com/paper/2403.09798/) (2024)
- [Self-Adaptive Cognitive Debiasing for Large Language Models in Decision-Making](https://mlsanity.com/paper/2504.04141/) (2025)

## Built on by (most cited first, 5 of 5)

- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Self-Improving Pretraining: using post-trained models to pretrain better models](https://mlsanity.com/paper/2601.21343/) (2026)
- [Characterizing Rhetorical Misalignment in Decision-Making with Language Models](https://mlsanity.com/paper/2608.14630/) (2026)
- [Deep and shallow biases in language models](https://mlsanity.com/paper/2609.09901/) (2026)
- [Monocultural Biases: Correlated biases in large language models lead to unequal systemic exclusion rates in hiring](https://mlsanity.com/paper/2609.22169/) (2026)

## Abstract (arXiv)

Large language models (LLMs) exhibit cognitive biases -- systematic tendencies of irrational decision-making, similar to those seen in humans. Prior work has found that these biases vary across models and can be amplified by instruction tuning. However, it remains unclear if these differences in biases stem from pretraining, finetuning, or even random noise due to training stochasticity. We propose a two-step causal experimental approach to disentangle these factors. First, we finetune models multiple times using different random seeds to study how training randomness affects over 30 cognitive biases. Second, we introduce cross-tuning -- swapping instruction datasets between models to isolate bias sources. This swap uses datasets that led to different bias patterns, directly testing whether biases are dataset-dependent. Our findings reveal that while training randomness introduces some variability, biases are mainly shaped by pretraining: models with the same pretrained backbone exhibit more similar bias patterns than those sharing only finetuning data. These insights suggest that understanding biases in finetuned models requires considering their pretraining origins beyond finetuning effects. This perspective can guide future efforts to develop principled strategies for evaluating and mitigating bias in LLMs.