mlsanityData to 7 Oct 2026

Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves

arXiv 2311.04205 ↗First posted 7 Nov 2023cs.CL cs.AI cs.LGcited by 123

Not on the map. Why?

Cited by 31 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Misunderstandings arise not only in interpersonal communication but also between humans and Large Language Models (LLMs). Such discrepancies can make LLMs interpret seemingly unambiguous questions in unexpected ways, yielding incorrect responses. While it is widely acknowledged that the quality of a prompt, such as a question, significantly impacts the quality of the response provided by LLMs, a systematic method for crafting questions that LLMs can better comprehend is still underdeveloped. In this paper, we present a method named `Rephrase and Respond' (RaR), which allows LLMs to rephrase and expand questions posed by humans and provide responses in a single prompt. This approach serves as a simple yet effective prompting method for improving performance. We also introduce a two-step variant of RaR, where a rephrasing LLM first rephrases the question and then passes the original and rephrased questions together to a different responding LLM. This facilitates the effective utilization of rephrased questions generated by one LLM with another. Our experiments demonstrate that our methods significantly improve the performance of different models across a wide range to tasks. We further provide a comprehensive comparison between RaR and the popular Chain-of-Thought (CoT) methods, both theoretically and empirically. We show that RaR is complementary to CoT and can be combined with CoT to achieve even better performance. Our work not only contributes to enhancing LLM performance efficiently and effectively but also sheds light on a fair evaluation of LLM capabilities. Data and codes are available at https://github.com/uclaml/Rephrase-and-Respond.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 47 of its 58 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

47 of 58 references.

References · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
35 more
  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  2. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  7. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  8. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  9. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  10. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  11. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  12. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  13. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  14. 2023Language Models can Solve Computer Taskscited by 371
  15. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  16. 2023How Language Model Hallucinations Can Snowballcited by 310
  17. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  18. 2022Generating Sequences by Learning to Self-Correctcited by 271
  19. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  20. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  21. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  22. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  23. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  24. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  25. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  26. 2022An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labelscited by 88
  27. 2023GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problemscited by 88
  28. 2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
  29. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
  30. 2023Selectively Answering Ambiguous Questionscited by 63
  31. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Modelscited by 61
  32. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52
  33. 2023Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Modelscited by 42
  34. 2023Learning to Simulate Natural Language Feedback for Interactive Semantic Parsingcited by 19
  35. 2023Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Modelscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 47; 2025: 50; 2026: 23.

123 papers.

Sort:

By area:

  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  2. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  3. 2024Distilling System 2 into System 1cited by 152
  4. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  5. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 117
  6. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 112
  7. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  8. 2024Approaching Human-Level Forecasting with Language Modelscited by 102
  9. 2023System 2 Attention (is something you might need too)cited by 85
  10. 2024Enhancing Large Vision Language Models with Self-Training on Image Comprehensioncited by 70
  11. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  12. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
38 more
  1. 2024TinyAgent: Function Calling at the Edgecited by 55
  2. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Modelscited by 48
  3. 2024Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Modelscited by 47
  4. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  5. 2024Thinking LLMs: General Instruction Following with Thought Generationcited by 45
  6. 2025Reasoning Language Models: A Blueprintcited by 41
  7. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  8. 2024Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLMcited by 38
  9. 2024Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Modelscited by 37
  10. 2025Self-Supervised Prompt Optimizationcited by 37
  11. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 34
  12. 2024Smoothie: Label Free Language Model Routingcited by 31
  13. 2024SPRIG: Improving Large Language Model Performance by System Prompt Optimizationcited by 28
  14. 2024HARP: A challenging human-annotated math reasoning benchmarkcited by 25
  15. 2024Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmarkcited by 22
  16. 2024A Survey of Conversational Searchcited by 22
  17. 2024Dipper: Diversity in Prompts for Producing Large Language Model Ensembles in Reasoning taskscited by 22
  18. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 21
  19. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  20. 2024PaCE: Parsimonious Concept Engineering for Large Language Modelscited by 19
  21. 2026Brittlebench: Quantifying LLM robustness via prompt sensitivitycited by 19
  22. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  23. 2023Social Learning: Towards Collaborative Learning with Large Language Modelscited by 15
  24. 2024Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queriescited by 15
  25. 2024Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary casescited by 13
  26. 2024Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Modelscited by 13
  27. 2024How Do Humans Write Code? Large Models Do It the Same Way Toocited by 11
  28. 2024LLM2: Let Large Language Models Harness System 2 Reasoningcited by 11
  29. 2025Syntriever: How to Train Your Retriever with Synthetic Data from LLMscited by 11
  30. 2025PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformationcited by 11
  31. 2023LLM-SQL-Solver: Can LLMs Determine SQL Equivalence?cited by 10
  32. 2024Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMscited by 10
  33. 2024Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimizationcited by 10
  34. 2024Merging Improves Self-Critique Against Jailbreak Attackscited by 8
  35. 2025Read Before You Think: Mitigating LLM Comprehension Failures with Step-by-Step Readingcited by 8
  36. 2025Bandit-Based Prompt Design Strategy Selection Improves Prompt Optimizerscited by 7
  37. 2024Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responsescited by 6
  38. 2024Content Knowledge Identification with Multi-Agent Large Language Models (LLMs)cited by 6

50 most cited of 123 All 123 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Putting People in LLMs' Shoes: Generating Better Answers via Question Rewriter
  2. 2024Rewriting Conversational Utterances with Instructed Large Language Models
  3. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questions
  4. 2023Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspection
  5. 2026ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA
  6. 2023Re-Reading Improves Reasoning in Large Language Models
  7. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responses
  8. 2026Feedback Indices to Evaluate LLM Responses to Rebuttals for Multiple Choice Type Questions