mlsanityData to 14 Oct 2026

Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation

arXiv 2609.36931 ↗First posted 29 Sep 2026cs.CLcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation also fits: Reasoning for agents, planning and code

Abstract (arXiv)

Reproducibility is essential for scientific research, yet prior work shows that LLM outputs vary with hardware and batching. We identify an overlooked factor: the hidden injection of the current date into system prompts, which users cannot control and which changes every day. Across 9 recent LLMs and 6 datasets spanning multiple-choice QA (MCQA), math reasoning, code generation, and machine translation, performance varies solely with the current date, with deltas of up to 6% on MCQA, 14% on math reasoning, 7% on code generation, and 2.84 BLEU on machine translation. Model rankings also shift, affecting leaderboards. This date effect exceeds other sources of non-determinism, such as batch size and numerical precision. Standard prompting techniques -- chain-of-thought and few-shot prompting -- do not reduce the sensitivity; chain-of-thought even amplifies it. Our findings underscore the need for careful evaluation protocols to ensure reproducibility and fair comparisons in LLM research.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 28 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

14 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility9 Apr

Other references · 13

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  7. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  8. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 116
  9. 2024Non-Determinism of "Deterministic" LLM Settingscited by 92
  10. 2025Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inferencecited by 53
  11. 2024How to Compute the Probability of a Wordcited by 36
  12. 2026Large Language Models Are Overconfident in Their Own Responsescited by 6
1 more
  1. 2026Calibration as a First-Class Criterion in LLM Evaluationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Investigating the Role of Reasoning-Language Alignment in Monolingual Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Does Prompt Formatting Have Any Impact on LLM Performance?
  2. 2025Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs
  3. 2025Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
  4. 2025Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
  5. 2024A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutions
  6. 2026Same Input, Different Scores: A Multi Model Study on the Inconsistency of LLM Judge
  7. 2026PromptResponse: Optimizing Prompts for LLM Coding Tasks
  8. 2026Aging of Prompt Engineering Techniques Across LLM Versions ↗