# Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understanding

arXiv: https://arxiv.org/abs/2404.04293 · HTML: https://mlsanity.com/paper/2404.04293/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 28 of 28)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [FOLIO: Natural Language Reasoning with First-Order Logic](https://mlsanity.com/paper/2209.00840/) (2022)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [Orca 2: Teaching Small Language Models How to Reason](https://mlsanity.com/paper/2311.11045/) (2023)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Learning From Mistakes Makes LLM Better Reasoner](https://mlsanity.com/paper/2310.20689/) (2023)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction](https://mlsanity.com/paper/2303.04132/) (2023)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020)
- [Can GPT-3 Perform Statutory Reasoning?](https://mlsanity.com/paper/2302.06100/) (2023)
- [LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning](https://mlsanity.com/paper/2007.08124/) (2020)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [DARE: Data Augmented Relation Extraction with GPT-2](https://mlsanity.com/paper/2004.13845/) (2020)
- [HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning](https://mlsanity.com/paper/1904.12166/) (2019)
- [The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks](https://mlsanity.com/paper/2304.13861/) (2023)
- [Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis](https://mlsanity.com/paper/2310.10477/) (2023)
- [TaxiNLI: Taking a Ride up the NLU Hill](https://mlsanity.com/paper/2009.14505/) (2020)
- [How susceptible are LLMs to Logical Fallacies?](https://mlsanity.com/paper/2308.09853/) (2023)
- [Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming](https://mlsanity.com/paper/2305.03742/) (2023)
- [Improving Large Language Models in Event Relation Logical Prediction](https://mlsanity.com/paper/2310.09158/) (2023)
- [GLoRE: Evaluating Logical Reasoning of Large Language Models](https://mlsanity.com/paper/2310.09107/) (2023)
- [LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models](https://mlsanity.com/paper/2203.15099/) (2022)
- [Case-Based Reasoning with Language Models for Classification of Logical Fallacies](https://mlsanity.com/paper/2301.11879/) (2023)

## Built on by (most cited first, 12 of 12)

- [Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities](https://mlsanity.com/paper/2406.02787/) (2024)
- [CoCoLoFa: A Dataset of News Comments with Common Logical Fallacies Written by LLM-Assisted Crowds](https://mlsanity.com/paper/2410.03457/) (2024)
- [A Logical Fallacy-Informed Framework for Argument Generation](https://mlsanity.com/paper/2408.03618/) (2024)
- [Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checks](https://mlsanity.com/paper/2503.00024/) (2025)
- [Emotionally Charged, Logically Blurred: AI-driven Emotional Framing Impairs Human Fallacy Detection](https://mlsanity.com/paper/2510.09695/) (2025)
- [Socrates or Smartypants: Testing Logic Reasoning Capabilities of Large Language Models with Logic Programming-based Test Oracles](https://mlsanity.com/paper/2504.12312/) (2025)
- [DuaShepherd: Integrating Stepwise Correctness and Potential Rewards for Mathematical Reasoning](https://mlsanity.com/paper/2506.17533/) (2025)
- [Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification](https://mlsanity.com/paper/2606.26698/) (2026)
- [A Few Hypocrites: Few-Shot Learning and Subtype Definitions for Detecting Hypocrisy Accusations in Online Climate Change Debates](https://mlsanity.com/paper/2409.16807/) (2024)
- [MisSynth: Improving MISSCI Logical Fallacies Classification with Synthetic Data](https://mlsanity.com/paper/2510.26345/) (2025)
- [Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentation](https://mlsanity.com/paper/2606.01020/) (2026)
- [ForEx: A Formal Verification Framework for Explainable Reasoning in Logical Fallacy Detection and Annotation](https://mlsanity.com/paper/2606.21867/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks, but they still struggle with some complicated reasoning tasks including logical reasoning. One non-negligible reason for LLMs' suboptimal performance on logical reasoning is their overlooking of understanding logical fallacies correctly. To evaluate LLMs' capability of logical fallacy understanding (LFU), we propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW in this paper. Towards these LFU tasks, we have successfully constructed a new dataset LFUD based on GPT-4 accompanied by a little human effort. Our extensive experiments justify that our LFUD can be used not only to evaluate LLMs' LFU capability, but also to fine-tune LLMs to obtain significantly enhanced performance on logical reasoning.