Data-driven Machine Learning Cannot Reach Symbolic-level Logical Reasoning -- The Limit of the Scaling Law
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
By promoting vectors to spheres and enabling explicit model construction, neural networks can perform symbolic-level syllogistic reasoning without training data. We identify two fundamental limitations that prevent conventional data-driven machine learning systems from achieving this capability: training data generated by the combination table cannot distinguish all 24 valid syllogism types, and end-to-end premise-to-conclusion mapping creates contradictory targets within neural components. Experiments with two representative conventional systems, GPT-5 using linguistic inputs and Euler Net using visual inputs, support this analysis. ChatGPT GPT-5 may reach 100% accuracy in syllogistic reasoning, but with hallucinations. Because the learning process terminates upon reaching 100% accuracy, the system cannot progress beyond empirical accuracy to symbolic level reasoning. Random test data reduced Euler Net's accuracy to 56%. Repeatedly expanding the training set increased its accuracy to 97%, with perfect performance on 8 syllogism types. However, because unintended inputs cannot be exhaustively covered, even 100% test accuracy does not imply symbolic-level reasoning. Since syllogistic reasoning underpins logical reasoning and human rationality, these results suggest that increasing data and training time alone cannot ensure symbolic level logical reasoning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 26 of its 53 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
26 of 53 references.
Closest ancestors · 6
Same direction, newest first
- 2026An AI Monkey Gets Grapes for Sure -- Sphere Neural Networks for Reliable Decision-Making1 Jan
- 2024Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset8 Aug
- 2024A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences17 Jun
- 2024Sphere Neural-Networks for Rational Reasoning22 Mar
- 2023A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models1 Nov
- 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May
Other references · 20
- 2017Attention Is All You Needcited by 60,361
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2023Let's Verify Step by Stepcited by 3,789
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
- 2021Explaining Neural Scaling Lawscited by 363
- 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
- 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
8 more
- 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
- 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 133
- 2023What Formal Languages Can Transformers Express? A Surveycited by 124
- 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Provingcited by 79
- 2023Siamese Masked Autoencoderscited by 76
- 2020Abstract Diagrammatic Reasoning with Multiplex Graph Networkscited by 37
- 2024SylloBio-NLI: Evaluating Large Language Models on Biomedical Syllogistic Reasoningcited by 9
- 2023On the Representational Capacity of Recurrent Neural Language Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Hybrid Models for Natural Language Reasoning: The Case of Syllogistic Logic
- 2023Empirical Investigation of Neural Symbolic Reasoning Strategies
- 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
- 2026UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning
- 2025Understanding Syllogistic Reasoning in LLMs from Formal and Natural Language Perspectives
- 2023Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems
- 2025Teaching Small Language Models to Learn Logic through Meta-Learning
- 2022Evaluating Step-by-Step Reasoning through Symbolic Verification