Judgment-of-Thought Prompting: A Courtroom-Inspired Framework for Binary Logical Reasoning with Large Language Models
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
This paper proposes a novel prompting approach, Judgment of Thought (JoT), specifically tailored for binary logical reasoning tasks. Despite advances in prompt engineering, existing approaches still face limitations in handling complex logical reasoning tasks. To address these issues, JoT introduces a multi-agent approach with three specialized rolesunicodex2010unicodex2010unicodex2010lawyer, prosecutor, and judgeunicodex2010unicodex2010unicodex2010where a high-level model acts as the judge, and lower-level models serve as lawyer and prosecutor to systematically debate and evaluate arguments. Experimental evaluations on benchmarks such as BigBenchHard and Winogrande demonstrate JoT's superior performance compared to existing prompting approaches, achieving notable improvements, including 98% accuracy in Boolean expressions. Also, our ablation studies validate the critical contribution of each role, iterative refinement loops, and feedback mechanisms. Consequently, JoT significantly enhances accuracy, reliability, and consistency in binary reasoning tasks and shows potential for practical applications.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 33 references that are in the map. First posted on arXiv 25 Sep 2024.
Builds on
11 of 33 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2023GPT-4 Technical Reportcited by 18,132
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2023A Survey of Large Language Modelscited by 2,537
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
- 2023Chain of Code: Reasoning with a Language Model-Augmented Code Emulatorcited by 120
- 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 115
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought
- 2024Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models
- 2024Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models
- 2024Logic Agent: Enhancing Validity with Logic Rule Invocation
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
- 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models
- 2025Theorem-of-Thought: A Multi-Agent Framework for Abductive, Deductive, and Inductive Reasoning in Language Models