mlsanityData to 7 Oct 2026

Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understanding

arXiv 2404.04293 ↗First posted 4 Apr 2024cs.CL cs.AIcited by 12

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks, but they still struggle with some complicated reasoning tasks including logical reasoning. One non-negligible reason for LLMs' suboptimal performance on logical reasoning is their overlooking of understanding logical fallacies correctly. To evaluate LLMs' capability of logical fallacy understanding (LFU), we propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW in this paper. Towards these LFU tasks, we have successfully constructed a new dataset LFUD based on GPT-4 accompanied by a little human effort. Our extensive experiments justify that our LFUD can be used not only to evaluate LLMs' LFU capability, but also to fine-tune LLMs to obtain significantly enhanced performance on logical reasoning.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 48 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

28 of 48 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023GLoRE: Evaluating Logical Reasoning of Large Language Models13 Oct
  2. 2023How susceptible are LLMs to Logical Fallacies?18 Aug
  3. 2023Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming5 May
  4. 2023Natural Language Reasoning, A Survey26 Mar
  5. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  6. 2023Case-Based Reasoning with Language Models for Classification of Logical Fallacies27 Jan
  7. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  8. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  4. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  5. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  6. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  7. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
  8. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  9. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
  10. 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
  11. 2020SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articlescited by 79
  12. 2023Can GPT-3 Perform Statutory Reasoning?cited by 71
8 more
  1. 2020LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoningcited by 68
  2. 2020DARE: Data Augmented Relation Extraction with GPT-2cited by 43
  3. 2019HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoningcited by 35
  4. 2023The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Taskscited by 35
  5. 2023Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysiscited by 34
  6. 2020TaxiNLI: Taking a Ride up the NLU Hillcited by 27
  7. 2023Improving Large Language Models in Event Relation Logical Predictioncited by 17
  8. 2022LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 3.

12 papers.

  1. 2024Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilitiescited by 22
  2. 2024CoCoLoFa: A Dataset of News Comments with Common Logical Fallacies Written by LLM-Assisted Crowdscited by 8
  3. 2024A Logical Fallacy-Informed Framework for Argument Generationcited by 7
  4. 2025Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checkscited by 6
  5. 2025Socrates or Smartypants: Testing Logic Reasoning Capabilities of Large Language Models with Logic Programming-based Test Oraclescited by 2
  6. 2025DuaShepherd: Integrating Stepwise Correctness and Potential Rewards for Mathematical Reasoningcited by 2
  7. 2025Emotionally Charged, Logically Blurred: AI-driven Emotional Framing Impairs Human Fallacy Detectioncited by 2
  8. 2026Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classificationcited by 1
  9. 2024A Few Hypocrites: Few-Shot Learning and Subtype Definitions for Detecting Hypocrisy Accusations in Online Climate Change Debatescited by 0
  10. 2025MisSynth: Improving MISSCI Logical Fallacies Classification with Synthetic Datacited by 0
  11. 2026Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentationcited by 0
  12. 2026ForEx: A Formal Verification Framework for Explainable Reasoning in Logical Fallacy Detection and Annotationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
  2. 2025Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs
  3. 2025Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation
  4. 2025Do Large Language Models Excel in Complex Logical Reasoning with Formal Language?
  5. 2025Logical Reasoning in Large Language Models: A Survey
  6. 2025Human-Level Reasoning: A Comparative Study of Large Language Models on Logical and Abstract Reasoning
  7. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
  8. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning