# Option-Order Randomisation Reveals a Distributional Position Attractor in Prompted Sandbagging

arXiv: https://arxiv.org/abs/2604.26206 · HTML: https://mlsanity.com/paper/2604.26206/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 16 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Builds on (most cited first, 13 of 13)

- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Probing and Steering Evaluation Awareness of Language Models](https://mlsanity.com/paper/2507.01786/) (2025)
- [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://mlsanity.com/paper/2412.01784/) (2024)
- [LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring](https://mlsanity.com/paper/2508.00943/) (2025)
- [Auditing Games for Sandbagging](https://mlsanity.com/paper/2512.07810/) (2025)
- [Set-LLM: A Permutation-Invariant LLM](https://mlsanity.com/paper/2505.15433/) (2025)
- [ABCD: All Biases Come Disguised](https://mlsanity.com/paper/2602.17445/) (2026)
- [Sandbagging in a Simple Survival Bandit Problem](https://mlsanity.com/paper/2509.26239/) (2025)
- [Below-Chance Blindness: Prompted Underperformance in Small LLMs Produces Positional Bias Rather than Answer Avoidance](https://mlsanity.com/paper/2604.25249/) (2026)

## Abstract (arXiv)

A predecessor pilot (Cacioli, 2026) found that Llama-3-8B implements prompted sandbagging as positional collapse rather than answer avoidance. However, fixed option ordering in MMLU-Pro left open whether this reflected a model-level position-dominant policy or dataset-level distractor structure. This pre-registered follow-up (3 models, 2,000 MMLU-Pro items, 4 conditions, 24,000 primary trials) added cyclic option-order randomisation as the critical control. The pre-registered item-level same-letter diagnostic did not confirm deterministic position-tracking (same-letter rate 37.3%, below the 50% threshold). However, pre-specified supporting analyses revealed that the response-position distribution under sandbagging was highly stable under complete content rotation (Pearson r = 0.9994; Jensen-Shannon divergence = 0.027, compared to 0.386 between honest and sandbagging conditions). Accuracy spiked to 72.1% when the correct answer coincidentally occupied the preferred position E, and fell to 4.3% at position A. The data provide strong evidence for a soft distributional attractor: under sandbagging instruction, the model enters a low-entropy response-position basin centred on E/F/G that is highly stable and largely content-invariant at the aggregate level. Qwen-2.5-7B served as a negative control (non-compliant, no distributional shift). These results provide evidence, at the 7-9 billion parameter scale, that response-position entropy is a promising black-box behavioural signature of this sandbagging mode.