mlsanityData to 7 Oct 2026

Asking Again and Again: Exploring LLM Robustness to Repeated Questions

arXiv 2412.07923 ↗First posted 10 Dec 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

This study investigates whether repeating questions within prompts influences the performance of large language models (LLMs). We hypothesize that reiterating a question within a single prompt might enhance the model's focus on key elements of the query. We evaluate five recent LLMs -- including GPT-4o-mini, DeepSeek-V3, and smaller open-source models -- on three reading comprehension datasets under different prompt settings, varying question repetition levels (1, 3, or 5 times per prompt). Our results demonstrate that question repetition can increase models' accuracy by up to 6%. However, across all models, settings, and datasets, we do not find the result statistically significant. These findings provide insights into prompt design and LLM behavior, suggesting that repetition alone does not significantly impact output quality.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

18 of 29 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  8. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  9. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  10. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  11. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  12. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 93
6 more
  1. 2024Collective Critics for Creative Story Generationcited by 21
  2. 2022TASA: Deceiving Question Answering Models by Twin Answer Sentences Attackcited by 15
  3. 2024Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citationscited by 15
  4. 2023EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learningcited by 13
  5. 2024MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generationcited by 13
  6. 2024It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehensioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Prompt Repetition Improves Non-Reasoning LLMscited by 23
  2. 2026Large Language Models Are Overconfident in Their Own Responsescited by 5
  3. 2025Mitigating Label Length Bias in Large Language Modelscited by 1
  4. 2026UpSkill: Mutual Information Skill Learning for Structured Response Diversity in LLMscited by 1
  5. 2025Mitigating Hallucinations in Zero-Shot Scientific Summarisation: A Pilot Studycited by 0