mlsanityData to 7 Oct 2026

Large Language Models Are Not Robust Multiple Choice Selectors

arXiv 2309.03882 ↗First posted 7 Sep 2023cs.CLcited by 454

Not on the map. Why?

Cited by 45 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs). This work shows that modern LLMs are vulnerable to option position changes in MCQs due to their inherent "selection bias", namely, they prefer to select specific option IDs as answers (like "Option A"). Through extensive empirical analyses with 20 LLMs on three benchmarks, we pinpoint that this behavioral bias primarily stems from LLMs' token bias, where the model a priori assigns more probabilistic mass to specific option ID tokens (e.g., A/B/C/D) when predicting answers from the option IDs. To mitigate selection bias, we propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution. PriDe first estimates the prior by permutating option contents on a small number of test samples, and then applies the estimated prior to debias the remaining samples. We demonstrate that it achieves interpretable and transferable debiasing with high computational efficiency. We hope this work can draw broader research attention to the bias and robustness of modern LLMs.

Key facts. Cited by 454 papers in the map (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 7 Sep 2023.

Builds on

25 of 30 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
13 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  5. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  6. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  7. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  8. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  9. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  10. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 149
  11. 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 71
  12. 2023Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrationscited by 49
  13. 2023Click: Controllable Text Generation with Sequence Likelihood Contrastive Learningcited by 41

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 138; 2025: 136; 2026: 172.

454 papers.

Sort:

By area:

  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  2. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  3. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  4. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  5. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  6. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  7. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  8. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  9. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
  10. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 146
  11. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Modelscited by 143
  12. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 138
38 more
  1. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 134
  2. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  3. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  4. 2024MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generationcited by 103
  5. 2024OLMES: A Standard for Language Model Evaluationscited by 100
  6. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  7. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  8. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  9. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 86
  10. 2024The Political Preferences of LLMscited by 83
  11. 2024A Closer Look at the Limitations of Instruction Tuningcited by 76
  12. 2024Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arenacited by 76
  13. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
  14. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluationcited by 73
  15. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
  16. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 68
  17. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Modelscited by 68
  18. 2024Eliminating Position Bias of Language Models: A Mechanistic Approachcited by 61
  19. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  20. 2024Large Language Models Reflect the Ideology of their Creatorscited by 56
  21. 2024Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMscited by 55
  22. 2024EmoBench: Evaluating the Emotional Intelligence of Large Language Modelscited by 53
  23. 2024Quantifying Variance in Evaluation Benchmarkscited by 52
  24. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  25. 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 49
  26. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
  27. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 43
  28. 2024Changing Answer Order Can Decrease MMLU Accuracycited by 41
  29. 2025XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQLcited by 40
  30. 2024Talking Heads: Understanding Inter-layer Communication in Transformer Language Modelscited by 39
  31. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 39
  32. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Modelscited by 39
  33. 2025Do LLM Evaluators Prefer Themselves for a Reason?cited by 38
  34. 2025Answer Matching Outperforms Multiple Choice for Language Model Evaluationcited by 37
  35. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 37
  36. 2025When2Call: When (not) to Call Toolscited by 36
  37. 2024Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questionscited by 35
  38. 2024CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judgescited by 32

50 most cited of 454 All 454 (JSON)