Understanding Syllogistic Reasoning in LLMs from Formal and Natural Language Perspectives
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
We study syllogistic reasoning in LLMs from the logical and natural language perspectives. In process, we explore fundamental reasoning capabilities of the LLMs and the direction this research is moving forward. To aid in our studies, we use 14 large language models and investigate their syllogistic reasoning capabilities in terms of symbolic inferences as well as natural language understanding. Even though this reasoning mechanism is not a uniform emergent property across LLMs, the perfect symbolic performances in certain models make us wonder whether LLMs are becoming more and more formal reasoning mechanisms, rather than making explicit the nuances of human reasoning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 3 of its 26 references that are in the map. First posted on arXiv 14 Dec 2025.
Builds on
3 of 26 references.
References · 3
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2023A Survey of Large Language Modelscited by 2,576
- 2023Universal Self-Consistency for Large Language Model Generationcited by 203
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences
- 2024Can Large Language Models Act as Symbolic Reasoners?
- 2024Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset
- 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
- 2026A Syllogistic Probe: Tracing the Evolution of Logic Reasoning in Large Language Models
- 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study
- 2025Logical forms complement probability in understanding language model (and human) performance
- 2024A Closer Look at Logical Reasoning with LLMs: The Choice of Tool Matters