mlsanityData to 7 Oct 2026

Logic Sketch Prompting (LSP): A Deterministic and Interpretable Prompting Method

arXiv 2512.22258 ↗First posted 24 Dec 2025cs.AI cs.LG cs.LOcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) excel at natural language reasoning but remain unreliable on tasks requiring strict rule adherence, determinism, and auditability. Logic Sketch Prompting (LSP) is a lightweight prompting framework that introduces typed variables, deterministic condition evaluators, and a rule based validator that produces traceable and repeatable outputs. Using two pharmacologic logic compliance tasks, we benchmark LSP against zero shot prompting, chain of thought prompting, and concise prompting across three open weight models: Gemma 2, Mistral, and Llama 3. Across both tasks and all models, LSP consistently achieves the highest accuracy (0.83 to 0.89) and F1 score (0.83 to 0.89), substantially outperforming zero shot prompting (0.24 to 0.60), concise prompts (0.16 to 0.30), and chain of thought prompting (0.56 to 0.75). McNemar tests show statistically significant gains for LSP across nearly all comparisons (p < 0.01). These results demonstrate that LSP improves determinism, interpretability, and consistency without sacrificing performance, supporting its use in clinical, regulated, and safety critical decision support systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 19 references that are in the map. First posted on arXiv 24 Dec 2025.

Builds on

7 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 5

  1. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  2. 2017Neural-Symbolic Learning and Reasoning: A Survey and Interpretationcited by 195
  3. 2022Faithful Reasoning Using Large Language Modelscited by 130
  4. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  5. 2023An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processingcited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs
  2. 2025You Don't Need Prompt Engineering Anymore: The Prompting Inversion
  3. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
  4. 2023Invalid Logic, Equivalent Gains: The Bizarreness of Reasoning in Language Model Prompting
  5. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  6. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
  7. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching
  8. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns