mlsanityData to 14 Oct 2026

When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA

arXiv 2609.08934 ↗First posted 8 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Language models often receive a question together with a claim about what another source answered. We audit whether such claims destabilize answers in multiple-choice question answering. For each item, we hold one wrong option fixed across misleading conditions and vary the cue template attached to it. We introduce neutral-conditioned misleading cue adoption rate (NC-MCAR), which measures switches to that option only on valid cued trials where the same model first selected the gold answer under a neutral prompt. This is a measure of answer instability, not proof that the model knew the answer or that all deference is irrational. We evaluate four instruction-following models on MMLU-Pro and IndicMMLU-Pro in English, Hindi, Bengali, Tamil, and Telugu. Across 220,000 outputs, the expert template yields 41.1% aggregate NC-MCAR, compared with 12.5% for the majority template. These two conditions use the same wrong option and final instruction. Filler accuracy remains well above expert-wrong accuracy, while correct-cue prompts have high valid-response accuracy. The audit documents answer instability relevant to grounding under the tested forced-choice prompts: a bare, unverified source claim can outweigh an answer that was previously consistent with the task evidence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 17 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

7 of 17 references.

References · 7

  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  2. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  3. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  4. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
  5. 2024Conformity in Large Language Modelscited by 34
  6. 2025Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answeringcited by 28
  7. 2025IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understandingcited by 10

Built on by

Not cited yet.