# Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation

arXiv: https://arxiv.org/abs/2503.23363 · HTML: https://mlsanity.com/paper/2503.23363/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 3 of its 29 references that are in the map.
- First posted on arXiv 30 Mar 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 3 of 3)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Good Questions Help Zero-Shot Image Reasoning](https://mlsanity.com/paper/2312.01598/) (2023)

## Built on by (most cited first, 10 of 10)

- [Emotionally Charged, Logically Blurred: AI-driven Emotional Framing Impairs Human Fallacy Detection](https://mlsanity.com/paper/2510.09695/) (2025)
- [Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs](https://mlsanity.com/paper/2510.09970/) (2025)
- [Critical Inker: Scaffolding Critical Thinking in AI-Assisted Writing Through Socratic Questioning](https://mlsanity.com/paper/2604.07167/) (2026)
- [Label Over Logic? How Source Cues Bias Human Fallacy Judgments More Than LLMs](https://mlsanity.com/paper/2605.29928/) (2026)
- [Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification](https://mlsanity.com/paper/2606.26698/) (2026)
- [Haibu Mathematical-Medical Intelligent Agent:Enhancing Large Language Model Reliability in Medical Tasks via Verifiable Reasoning Chains](https://mlsanity.com/paper/2510.07748/) (2025)
- [Can One-sided Arguments Lead to Response Change in Large Language Models?](https://mlsanity.com/paper/2602.06260/) (2026)
- [Policy Compliance of User Requests in Natural Language for AI Systems](https://mlsanity.com/paper/2603.00369/) (2026)
- [Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentation](https://mlsanity.com/paper/2606.01020/) (2026)
- [Retrieving Relations, Detecting Fallacies: A RAG Approach to Political Debate Analysis](https://mlsanity.com/paper/2608.27471/) (2026)

## Abstract (arXiv)

The advancement of Large Language Models (LLMs) has greatly improved our ability to process complex language. However, accurately detecting logical fallacies remains a significant challenge. This study presents a novel and effective prompt formulation approach for logical fallacy detection, applicable in both supervised (fine-tuned) and unsupervised (zero-shot) settings. Our method enriches input text incorporating implicit contextual information -- counterarguments, explanations, and goals -- which we query for validity within the context of the argument. We then rank these queries based on confidence scores to inform classification. We evaluate our approach across multiple datasets from 5 domains, covering 29 distinct fallacy types, using models from the GPT and LLaMA series. The results show substantial improvements over state-of-the-art models, with F1 score increases of up to 0.60 in zero-shot settings and up to 0.45 in fine-tuned models. Extensive analyses further illustrate why and how our method excels.