mlsanityData to 13 Oct 2026

Language Models Benefit from Preparation with Elicited Knowledge

arXiv 2409.01345 ↗First posted 2 Sep 2024cs.CL cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The zero-shot chain of thought (CoT) approach is often used in question answering (QA) by language models (LMs) for tasks that require multiple reasoning steps. However, some QA tasks hinge more on accessing relevant knowledge than on chaining reasoning steps. We introduce a simple prompting technique, called PREP, that involves using two instances of LMs: the first (LM1) generates relevant information, and the second (LM2) receives the information from the user and answers the question. This design is intended to make better use of the LM's instruction-following capability. PREP is applicable across various QA tasks without domain-specific prompt engineering. PREP is developed on a dataset of 100 QA questions, derived from an extensive schematic dataset specifying artifact parts and material composition. These questions ask which of two artifacts is less likely to share materials with another artifact. Such questions probe the LM's knowledge of shared materials in the part structure of different artifacts. We test our method on our parts-and-materials dataset and three published commonsense reasoning datasets. The average accuracy of our method is consistently higher than that of all the other tested methods across all the tested datasets.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 13 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

6 of 13 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Large Language Models are Zero-Shot Reasoners24 May

Other references · 5

  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,133
  2. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,805
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 738
  4. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  5. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 123

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  2. 2026Evaluating Prompting Strategies for Chart Question Answering with Large Language Models
  3. 2023Semi-Structured Chain-of-Thought: Integrating Multiple Sources of Knowledge for Improved Language Model Reasoning
  4. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  5. 2023Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspection
  6. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
  7. 2025SQuARE: Sequential Question Answering Reasoning Engine for Enhanced Chain-of-Thought in Large Language Models
  8. 2026ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA