DeltaLogic: Minimal Premise Edits Reveal Belief-Revision Failures in Logical Reasoning Models
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Reasoning benchmarks typically evaluate whether a model derives the correct answer from a fixed premise set, but they under-measure a closely related capability that matters in dynamic environments: belief revision under minimal evidence change. We introduce DeltaLogic, a benchmark transformation protocol that converts natural-language reasoning examples into short revision episodes. Each episode first asks for an initial conclusion under premises P, then applies a minimal edit δ(P), and finally asks whether the previous conclusion should remain stable or be revised. We instantiate DeltaLogic from FOLIO and ProofWriter and evaluate small causal language models with constrained label scoring. On a completed 30-episode Qwen evaluation subset, stronger initial reasoning still does not imply stronger revision behavior: Qwen3-1.7B reaches 0.667 initial accuracy but only 0.467 revision accuracy, with inertia rising to 0.600 on episodes where the gold label should change, while Qwen3-0.6B collapses into near universal abstention. There, Qwen3-4B preserves the same inertial failure pattern (0.650 initial, 0.450 revised, 0.600 inertia), whereas Phi-4-mini-instruct is substantially stronger (0.950 initial, 0.850 revised) but still exhibits non-trivial abstention and control instability. These results suggest that logical competence under fixed premises does not imply disciplined belief revision after local evidence edits. DeltaLogic therefore targets a distinct and practically important reasoning capability that complements existing logical inference and belief-updating benchmarks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 5 of its 7 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
5 of 7 references.
Closest ancestors · 1
Same direction, newest first
Other references · 4
- 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
- 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
- 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 67
- 2024LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Modelscited by 19
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026A Single Revision Step Improves Token-Efficient LLM Reasoning
- 2026Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
- 2023SCREWS: A Modular Framework for Reasoning with Revisions
- 2026Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores
- 2025RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation
- 2026AERA: Adaptive Evidence Residual Allocation for Efficient Test-Time Reasoning
- 2026ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models
- 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study