mlsanityData to 7 Oct 2026

Using Language Models For Knowledge Acquisition in Natural Language Reasoning Problems

arXiv 2304.01771 ↗First posted 4 Apr 2023cs.AIcited by 2

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

For a natural language problem that requires some non-trivial reasoning to solve, there are at least two ways to do it using a large language model (LLM). One is to ask it to solve it directly. The other is to use it to extract the facts from the problem text and then use a theorem prover to solve it. In this note, we compare the two methods using ChatGPT and GPT4 on a series of logic word puzzles, and conclude that the latter is the right approach.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 8 of its 20 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

8 of 20 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
  2. 2022Faithful Reasoning Using Large Language Models30 Aug
  3. 2022On the Paradox of Learning to Reason from Data23 May
  4. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May

Other references · 4

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Teaching Temporal Logics to Neural Networkscited by 51
  4. 2020Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledgecited by 24

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Conceptual In-Context Learning and Chain of Concepts: Solving Complex Conceptual Problems Using Large Language Modelscited by 2
  2. 2025PCRLLM: Proof-Carrying Reasoning with Large Language Models under Stepwise Logical Constraintscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
  2. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate
  3. 2025Large Language Models' Reasoning Stalls: An Investigation into the Capabilities of Frontier Models
  4. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  5. 2025Improving LLMs' Generalized Reasoning Abilities by Graph Problems
  6. 2025Large Language Models and Mathematical Reasoning Failures
  7. 2024General Purpose Verification for Chain of Thought Prompting
  8. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4