# Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks

arXiv: https://arxiv.org/abs/2407.03624 · HTML: https://mlsanity.com/paper/2407.03624/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 8 of 8)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models](https://mlsanity.com/paper/2204.01172/) (2022)
- [A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models](https://mlsanity.com/paper/2307.12980/) (2023)

## Built on by (most cited first, 13 of 13)

- [MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability](https://mlsanity.com/paper/2505.20285/) (2025)
- [Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries](https://mlsanity.com/paper/2503.11256/) (2025)
- [Evaluation of LLMs for mathematical problem solving](https://mlsanity.com/paper/2506.00309/) (2025)
- [KnowRL: Teaching Language Models to Know What They Know](https://mlsanity.com/paper/2510.11407/) (2025)
- [BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactions](https://mlsanity.com/paper/2506.05766/) (2025)
- [Investigating Symbolic Triggers of Hallucination in Gemma Models Across HaluEval and TruthfulQA](https://mlsanity.com/paper/2509.09715/) (2025)
- [Balancing Content Size in RAG-Text2SQL System](https://mlsanity.com/paper/2502.15723/) (2025)
- [Mirage of Mastery: Memorization Tricks LLMs into Artificially Inflated Self-Knowledge](https://mlsanity.com/paper/2506.18998/) (2025)
- [From Test-taking to Cognitive Scaffolding: A Pedagogical Diagnostic Benchmark for LLMs on English Standardized Tests](https://mlsanity.com/paper/2505.17056/) (2025)
- [MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing](https://mlsanity.com/paper/2603.22289/) (2026)
- [Are Humans as Brittle as Large Language Models?](https://mlsanity.com/paper/2509.07869/) (2025)
- [Active Confusion Expression in Large Language Models: Leveraging World Models toward Better Social Reasoning](https://mlsanity.com/paper/2510.07974/) (2025)
- [iPOE: Interpretable Prompt Optimization via Explanations](https://mlsanity.com/paper/2605.18113/) (2026)

## Abstract (arXiv)

Although LLMs have the potential to transform many fields, they still underperform humans in reasoning tasks. Existing methods induce the model to produce step-by-step calculations, but this research explores the question: Does making the LLM analyze the question improve its performance? We propose a novel prompting strategy called Question Analysis Prompting (QAP), in which the model is prompted to explain the question in n words before solving. The value of n influences the length of response generated by the model. QAP is evaluated on GPT 3.5 Turbo and GPT 4 Turbo on arithmetic datasets GSM8K, AQuA, and SAT and commonsense dataset StrategyQA. QAP is compared with other state-of-the-art prompts including Chain-of-Thought (CoT), Plan and Solve Prompting (PS+) and Take A Deep Breath (TADB). QAP outperforms all state-of-the-art prompts on AQuA and SAT datasets on both GPT3.5 and GPT4. QAP consistently ranks among the top-2 prompts on 75% of the tests. A key factor of QAP performance can be attributed to response length, where detailed responses are beneficial when answering harder questions, but can negatively affect easy questions.