A Syllogistic Probe: Tracing the Evolution of Logic Reasoning in Large Language Models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Human logic has gradually shifted from intuition-driven inference to rigorous formal systems. Motivated by recent advances in large language models (LLMs), we explore whether LLMs exhibit a similar evolution in the underlying logical framework. Using existential import as a probe, we for evaluate syllogism under traditional and modern logic. Through extensive experiments of testing SOTA LLMs on a new syllogism dataset, we have some interesting findings: (i) Model size scaling promotes the shift toward modern logic; (ii) Thinking serves as an efficient accelerator beyond parameter scaling; (iii) the Base model plays a crucial role in determining how easily and stably this shift can emerge. Beyond these core factors, we conduct additional experiments for in-depth analysis of properties of current LLMs on syllogistic reasoning.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 21 of its 34 references that are in the map. First posted on arXiv 24 Jan 2026.
Builds on
21 of 34 references.
Closest ancestors · 5
Same direction, newest first
- 2025BIS Reasoning 1.0: The First Large-Scale Japanese Benchmark for Belief-Inconsistent Syllogistic Reasoning8 Jun
- 2025Qwen3 Technical Report14 May
- 2023Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases21 Jun
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity8 Feb
- 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
Other references · 16
- 2024The Llama 3 Herd of Modelscited by 14,486
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,247
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,714
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,843
- 2015A large annotated corpus for learning natural language inferencecited by 1,841
- 2025Gemma 3 Technical Reportcited by 1,748
- 2025Large Language Diffusion Modelscited by 888
- 2025Humanity's Last Examcited by 478
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 275
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 161
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 121
- 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 111
4 more
- 2023CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?cited by 89
- 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 84
- 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
- 2020Learning Syllogism with Euler Neural-Networkscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences
- 2024Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset
- 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
- 2025Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework
- 2026Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models
- 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
- 2025Hybrid Models for Natural Language Reasoning: The Case of Syllogistic Logic
- 2024Conditional and Modal Reasoning in Large Language Models