# Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models

arXiv: https://arxiv.org/abs/2506.14092 · HTML: https://mlsanity.com/paper/2506.14092/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 98 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 30 of 40)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2406.07791/) (2024)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [The Effect of Sampling Temperature on Problem Solving in Large Language Models](https://mlsanity.com/paper/2402.05201/) (2024)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [Premise Order Matters in Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08939/) (2024)
- [Large language models surpass human experts in predicting neuroscience results](https://mlsanity.com/paper/2403.03230/) (2024)
- [On the Emergence of Position Bias in Transformers](https://mlsanity.com/paper/2502.01951/) (2025)
- [Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination](https://mlsanity.com/paper/2406.08818/) (2024)
- [A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models](https://mlsanity.com/paper/2310.09497/) (2023)
- [Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval](https://mlsanity.com/paper/2407.20371/) (2024)
- [LLM Voting: Human Choices and AI Collective Decision Making](https://mlsanity.com/paper/2402.01766/) (2024)
- [Anchoring Bias in Large Language Models: An Experimental Study](https://mlsanity.com/paper/2412.06593/) (2024)
- [Comparing Biases and the Impact of Multilingual Training across Multiple Languages](https://mlsanity.com/paper/2305.11242/) (2023)
- [A Comprehensive Evaluation of Cognitive Biases in LLMs](https://mlsanity.com/paper/2410.15413/) (2024)
- [Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey](https://mlsanity.com/paper/2309.13933/) (2023)
- [Serial Position Effects of Large Language Models](https://mlsanity.com/paper/2406.15981/) (2024)
- [Large Language Models are overconfident and amplify human bias](https://mlsanity.com/paper/2505.02151/) (2025)
- [The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring](https://mlsanity.com/paper/2405.04412/) (2024)
- [Primacy Effect of ChatGPT](https://mlsanity.com/paper/2310.13206/) (2023)
- [Open (Clinical) LLMs are Sensitive to Instruction Phrasings](https://mlsanity.com/paper/2407.09429/) (2024)

## Built on by (most cited first, 5 of 5)

- [Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment](https://mlsanity.com/paper/2602.14777/) (2026)
- [Generalization of Fine-Tuned Uncertainty Communication and Metacognition in Large Language Models](https://mlsanity.com/paper/2510.05126/) (2025)
- [Google's AI & Economy ATLAS v1.0: Mapping Gemini Usage in the Economy](https://mlsanity.com/paper/2608.00038/) (2026)
- [Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders](https://mlsanity.com/paper/2609.17989/) (2026)
- [Epistemic Warrant for LLM Recommendations: Characterizing the Basis for Reliance When Ground Truth Is Unavailable](https://mlsanity.com/paper/2609.04127/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly deployed in decision-support systems for high-stakes domains such as hiring and university admissions, where choices often involve selecting among competing alternatives. While prior work has noted position biases in LLM-driven comparisons, these biases have not been systematically analyzed or linked to underlying preference structures. We present the first comprehensive study of position biases across multiple LLMs and two distinct domains: resume comparisons, representing a realistic high-stakes context, and color selection, which isolates position effects by removing confounding factors. We find strong and consistent order effects, including a quality-dependent shift: when all options are high quality, models favor the first option, but when quality is lower, they favor later options. We also identify a previously undocumented bias: a name bias, where certain names are favored despite controlling for demographic signals. To separate superficial tie-breaking from genuine distortions of judgment, we extend the rational choice framework to classify pairwise preferences as robust, fragile, or indifferent. Using this framework, we show that order effects can lead models to select strictly inferior options. These results indicate that LLMs exhibit distinct failure modes not documented in human decision-making. We also propose targeted mitigation strategies, including a novel use of the temperature parameter, to recover underlying preferences when order effects distort model behavior.