mlsanityData to 7 Oct 2026

ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs

arXiv 2603.02676 ↗First posted 3 Mar 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models suffer from content effects in reasoning tasks, particularly in multi-lingual contexts. We introduce a novel method that reduces these biases through explicit structural abstraction that transforms syllogisms into canonical logical representations and applies deterministic parsing to determine validity. Evaluated on the SemEval-2026 Task 11 multilingual benchmark, our approach achieves top-5 rankings across all subtasks while substantially reducing content effects and offering a competitive alternative to complex fine-tuning or activation-level interventions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 13 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

10 of 13 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions18 Feb
  2. 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference16 Aug
  3. 2024Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset8 Aug
  4. 2024Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism26 Jun

Other references · 6

  1. 2024Faithful Logical Reasoning via Symbolic Chain-of-Thoughtcited by 99
  2. 2024Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Provingcited by 35
  3. 2024Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMscited by 19
  4. 2025Mitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steeringcited by 18
  5. 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarkscited by 8
  6. 2025Language Surgery in Multilingual Large Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy
  2. 2026HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization
  3. 2025Do Large Language Models Excel in Complex Logical Reasoning with Formal Language?
  4. 2025Are LLMs Stable Formal Logic Translators in Logical Reasoning Across Linguistically Diversified Texts?
  5. 2024Eliciting Better Multilingual Structured Reasoning from LLMs through Code
  6. 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study
  7. 2025LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning
  8. 2026QMFOL: Benchmarking Large Language Model Reasoning via Quantifiable Monadic First-Order Logic Test Case Generation