mlsanityData to 14 Oct 2026

The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context

arXiv 2607.12963 ↗First posted 14 Jul 2026cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of-the-art models often appear robust to task-irrelevant context at the aggregate level: prepending it to benchmark questions causes little change in overall accuracy. This aggregate stability, however, masks significant per-example instability. Even semantically meaningless pseudo-words, formed by randomly combining characters, can markedly shift model predictions on a small fraction of examples, degrading performance on some while improving it on others. This two-sided effect holds consistently across a wide range of models and datasets, yet the affected examples are largely model-specific. We further show that this instability is modulated by context type, context length, test-time compute, and model development stage. Together, our findings reveal context-induced tail risks concealed by aggregate accuracy, motivating per-example reliability evaluation of language models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 32 of its 35 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

32 of 35 references.

References · 32

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  9. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  10. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  11. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
20 more
  1. 2024Measuring short-form factuality in large language modelscited by 373
  2. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  3. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  4. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  5. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  6. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 160
  7. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 152
  8. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 149
  9. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 127
  10. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 100
  11. 2026Towards a Science of AI Agent Reliabilitycited by 65
  12. 2025SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledgecited by 54
  13. 2025How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmarkcited by 28
  14. 2025Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Searchcited by 10
  15. 2024Reducing Distraction in Long-Context Language Models by Focused Learningcited by 9
  16. 2025Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoningcited by 8
  17. 2026A Bitter Lesson for Data Filteringcited by 8
  18. 2025Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMscited by 6
  19. 2026Reasoning Shift: How Context Silently Shortens LLM Reasoningcited by 3
  20. 2026Better and Worse with Scale: How Contextual Entrainment Diverges with Model Sizecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026When benchmark inferences do not compose: Projectibility in AI evaluationcited by 0
  2. 2026What Will Post-Training Fix? Per-Problem Gains Are Shared Across Independent RL Runs, and Existing Checkpoints Predict Them Better Than A Priori Signalscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets
  2. 2023Self-Consistency of Large Language Models under Ambiguity
  3. 2026Context Dependence and Reliability in Autoregressive Language Models
  4. 2024Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
  5. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
  6. 2026Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity
  7. 2026On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
  8. 2026Context-Length Robustness in Question Answering Models: A Comparative Empirical Study