mlsanityData to 7 Oct 2026

Compositional Consistency-Guided Decoding for Three-Way Logical Question Answering

arXiv 2604.06196 ↗First posted 12 Mar 2026cs.CL cs.AI cs.LOcited by 0

Not on the map. Why?

Abstract (arXiv)

Three-way logical question answering (QA) assigns one of True, False, or Unknown to a hypothesis H given a premise set S. We study this task as a compact compositional inference problem: predictions for H and for a mechanically negated hypothesis neg H should agree under a deterministic negation map. Despite this simple structure, large language models (LLMs) can exhibit two practical failure modes: (i) negation inconsistency, where answers to H and neg H violate the required label mapping, and (ii) epistemic Unknown, where the model abstains even when one side is entailed. We introduce CGD-PD, a lightweight, training-free test-time layer that combines neural 3-way classification, symbolic negation-consistency projection, and targeted binary entailment probes. On one validation split of FOLIO's first-order logic fields, CGD-PD improves accuracy by 4.4 points on GPT-5.2 and 6.8 points on Claude Sonnet 4.5, while reducing Unknown predictions and epistemic abstention. These results provide a controlled proof of concept that simple logical composition at inference time can help evaluate and improve LLM reasoning reliability; they do not, by themselves, establish robustness beyond this formal benchmark setting.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

15 of 21 references.

References · 15

  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  2. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  6. 2017Selective Classification for Deep Neural Networkscited by 799
  7. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  8. 2015Neural Module Networkscited by 375
  9. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  10. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  11. 2020Transformers as Soft Reasoners over Languagecited by 265
  12. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 234
3 more
  1. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 181
  2. 2024P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chainscited by 13
  3. 2025Metamorphic Testing of Large Language Models for Natural Language Processingcited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Controlled Reformulation Testing for Logical Consistency in Large Language Models
  2. 2026From Atomic Evidence to Logical Composition: Structured Compositional Reasoning over Compound Answer Options
  3. 2026Operadic consistency: a label-free signal for compositional reasoning failures in LLMs
  4. 2025Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning
  5. 2022Measuring Compositional Consistency for Video Question Answering
  6. 2026PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuning
  7. 2022Faithful Reasoning Using Large Language Models
  8. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models