UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbolic approach, combining a symbolic prover with small reasoning LLMs (4B parameters). The system consists of an LLM-based parser that translates natural language syllogisms to a first-order logic (FOL) representation, an automated theorem prover, and two optional modules: machine translation for multilingual inputs and a symbolic retrieval component for the identification of relevant premises. The system achieves competitive accuracy and relatively low content effect on most subtasks. Our ablations show that this approach outperforms LLM-based zero-shot baselines in this parameter size range, but also reveal limited multilingual capabilities of small LLMs. Finally, we include a discussion of the task's main ranking metric and analyze its limitations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 4 of its 21 references that are in the map. First posted on arXiv 6 May 2026.
Builds on
4 of 21 references.
Closest ancestors · 2
Same direction, newest first
- 2025Qwen3 Technical Report14 May
- 2024LOGIC-LM++: Multi-Step Refinement for Symbolic Formulations22 Jun
Other references · 2
- 2025Gemma 3 Technical Reportcited by 1,824
- 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
- 2024Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach
- 2025Hybrid Models for Natural Language Reasoning: The Case of Syllogistic Logic
- 2026Logic-Parametric Neuro-Symbolic NLI: Controlling Logical Formalisms for Verifiable LLM Reasoning
- 2025Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition
- 2026From Natural Language to Executable Narsese: A Neuro-Symbolic Benchmark and Pipeline for Reasoning with NARS
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers
- 2025Intermediate Languages Matter: Formal Languages and LLMs affect Neurosymbolic Reasoning