A criterion for Artificial General Intelligence: hypothetic-deductive reasoning, tested on ChatGPT
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
We argue that a key reasoning skill that any advanced AI, say GPT-4, should master in order to qualify as 'thinking machine', or AGI, is hypothetic-deductive reasoning. Problem-solving or question-answering can quite generally be construed as involving two steps: hypothesizing that a certain set of hypotheses T applies to the problem or question at hand, and deducing the solution or answer from T - hence the term hypothetic-deductive reasoning. An elementary proxy of hypothetic-deductive reasoning is causal reasoning. We propose simple tests for both types of reasoning, and apply them to ChatGPT. Our study shows that, at present, the chatbot has a limited capacity for either type of reasoning, as soon as the problems considered are somewhat complex. However, we submit that if an AI would be capable of this type of reasoning in a sufficiently wide range of contexts, it would be an AGI.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 18 of its 45 references that are in the map. First posted on arXiv 5 Aug 2023.
Builds on
18 of 45 references.
Closest ancestors · 3
Same direction, newest first
- 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-47 Apr
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity8 Feb
- 2022Towards Reasoning in Large Language Models: A Survey20 Dec
Other references · 15
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
- 2023Mathematical Capabilities of ChatGPTcited by 268
- 2023A Categorical Archive of ChatGPT Failurescited by 168
- 2023Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physicscited by 49
- 2023Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosiscited by 36
- 2023When do you need Chain-of-Thought Prompting for ChatGPT?cited by 35
3 more
Built on by
Citing papers per year: 2023: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Recognizing Artificial Minds: A Philosophical Defense of AI Cognition
- 2025Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence
- 2025Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impact
- 2025A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning
- 2023A Survey of Reasoning with Foundation Models
- 2025From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models
- 2023Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models
- 2025General Intelligence Requires Reward-based Pretraining