mlsanityData to 7 Oct 2026

Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation

arXiv 2604.27249 ↗First posted 29 Apr 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

When instructed to underperform on multiple-choice evaluations, do language models engage with question content or fall back on positional shortcuts? We map the boundary between these regimes using a six-condition adversarial instruction-specificity gradient administered to two instruction-tuned LLMs (Llama-3-8B and Llama-3.1-8B) on 2,000 MMLU-Pro items. Distributional screening (response-position entropy) and an independent content-engagement criterion (difficulty-accuracy correlation) jointly characterise each condition. The gradient reveals three regimes rather than a monotonic transition. Vague adversarial instructions produce moderate accuracy reduction with preserved content engagement. Standard sandbagging and capability-imitation instructions produce positional entropy collapse with partial content engagement. A two-step answer-aware avoidance instruction produces extreme positional collapse, with near-total concentration on a single response position (99.9% and 87.4%) and no measurable content sensitivity. This was the only multi-step instruction tested, and it produced the most extreme shortcut. The attractor position matches each model's content-absent null-prompt default. The effect replicates across both models and four academic domains. Distributional collapse and content engagement can co-occur (50% concordance between screening criteria), indicating that entropy-based screening and difficulty-based content assessment capture partially independent dimensions of response validity. Results suggest that instruction complexity can determine whether adversarial compliance uses content-aware or content-blind mechanisms in small instruction-tuned LLMs under greedy decoding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 16 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

9 of 16 references.

References · 9

  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  2. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  3. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  4. 2024Frontier Models are Capable of In-context Schemingcited by 252
  5. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  6. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  7. 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
  8. 2025LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoringcited by 17
  9. 2025Separating Constraint Compliance from Semantic Accuracy: A Novel Benchmark for Evaluating Instruction-Following Under Compressioncited by 1

Built on by

Not cited yet.