mlsanityData to 7 Oct 2026

DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models

arXiv 2512.02246 ↗First posted 1 Dec 2025cs.CL cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Prompt design plays a critical role in the reasoning performance of large language models (LLMs), yet the impact of prompt specificity - how detailed or vague a prompt is - remains understudied. This paper introduces DETAIL, a framework for evaluating LLM performance across varying levels of prompt specificity. We generate multi-level prompts using GPT-4, quantify specificity via perplexity, and assess correctness using GPT-based semantic equivalence. Experiments on 30 novel reasoning tasks across GPT-4 and O3-mini reveal that specificity improves accuracy, especially for smaller models and procedural tasks. Our results highlight the need for adaptive prompting strategies and provide tools and data to support further research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 30 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

20 of 30 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning23 May
  2. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models6 May
  3. 2022Large Language Models are Zero-Shot Reasoners24 May
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 16

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  9. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  10. 2025Logical Reasoning in Large Language Models: A Surveycited by 52
  11. 2025Empowering LLMs with Logical Reasoning: A Comprehensive Surveycited by 51
  12. 2025Do Large Language Model Benchmarks Test Reliability?cited by 50
4 more
  1. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approachescited by 43
  2. 2024Codenames as a Benchmark for Large Language Modelscited by 8
  3. 2023Sensing of inspiration events from speech: comparison of deep learning and linguistic methodscited by 1
  4. 2023Unicode Normalization and Grapheme Parsing of Indic Languagescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Metacognitive Prompting Improves Understanding in Large Language Models
  2. 2026Beyond the Prompt in Large Language Models: Comprehension, In-Context Learning, and Chain-of-Thought
  3. 2025Exploring LLM Reasoning Through Controlled Prompt Variations
  4. 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
  5. 2025Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs
  6. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  7. 2025Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages
  8. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt