mlsanityData to 7 Oct 2026

SEF-CLGC at SemEval-2026 Task 11: Logical Notation Impact on Language Model Performance

arXiv 2606.09157 ↗First posted 8 Jun 2026cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper revisits our pipeline called Syllogistic Evaluation Framework-Common Logic Grammar Construction (SEF-CLGC). We combine formal logical notations with Small Language Models (SLMs) to evaluate reasoning performance on the SemEval-2026 Task 11 Subtask 1: Disentangling Content and Formal Reasoning in Large Language Models. Our experiments show that by relying solely on SLMs, trained on a combination of natural and symbolic languages, our best model achieves a content score of 27.80% on the task while significantly lowering the content bias in reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

14 of 22 references.

References · 14

  1. 2024Faithful Logical Reasoning via Symbolic Chain-of-Thoughtcited by 99
  2. 2014Clingo = ASP + Control: Preliminary Reportcited by 56
  3. 2024Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Provingcited by 35
  4. 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inferencecited by 28
  5. 2025Mitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steeringcited by 18
  6. 2025Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractionscited by 17
  7. 2024Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Datasetcited by 14
  8. 2025Guiding Reasoning in Small Language Models with LLM Assistancecited by 13
  9. 2024SylloBio-NLI: Evaluating Large Language Models on Biomedical Syllogistic Reasoningcited by 9
  10. 2025Slm-mux: Orchestrating small language models for reasoningcited by 5
  11. 2025Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Searchcited by 3
  12. 2026Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Modelscited by 3
2 more
  1. 2025Investigating Language Model Capabilities to Represent and Process Formal Knowledge: A Preliminary Study to Assist Ontology Engineeringcited by 2
  2. 2026Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classificationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Are you Talking Logic to Me? Assessing Language Models Syllogistic Reasoning Capabilitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LogicSkills: A Structured Benchmark for Formal Reasoning in Large Language Models
  2. 2025Do Large Language Models Excel in Complex Logical Reasoning with Formal Language?
  3. 2024LOGIC-LM++: Multi-Step Refinement for Symbolic Formulations
  4. 2026Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs
  5. 2026QMFOL: Benchmarking Large Language Model Reasoning via Quantifiable Monadic First-Order Logic Test Case Generation
  6. 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study
  7. 2025Are LLMs Stable Formal Logic Translators in Logical Reasoning Across Linguistically Diversified Texts?
  8. 2026ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs