# Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling

arXiv: https://arxiv.org/abs/2406.12402 · HTML: https://mlsanity.com/paper/2406.12402/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 33 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 5 of 5)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Argotario: Computational Argumentation Meets Serious Games](https://mlsanity.com/paper/1707.06002/) (2017)
- [Multitask Instruction-based Prompting for Fallacy Recognition](https://mlsanity.com/paper/2301.09992/) (2023)
- [Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments](https://mlsanity.com/paper/2212.07425/) (2022)

## Built on by (most cited first, 3 of 3)

- [Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification](https://mlsanity.com/paper/2606.26698/) (2026)
- [Can One-sided Arguments Lead to Response Change in Large Language Models?](https://mlsanity.com/paper/2602.06260/) (2026)
- [Truth or Sophistry? LoFa: A Benchmark for LLM Robustness Against Logical Fallacies](https://mlsanity.com/paper/2606.31039/) (2026)

## Abstract (arXiv)

Prior research in computational argumentation has mainly focused on scoring the quality of arguments, with less attention on explicating logical errors. In this work, we introduce four sets of explainable templates for common informal logical fallacies designed to explicate a fallacy's implicit logic. Using our templates, we conduct an annotation study on top of 400 fallacious arguments taken from LOGIC dataset and achieve a high agreement score (Krippendorf's alpha of 0.54) and reasonable coverage (0.83). Finally, we conduct an experiment for detecting the structure of fallacies and discover that state-of-the-art language models struggle with detecting fallacy templates (0.47 accuracy). To facilitate research on fallacies, we make our dataset and guidelines publicly available.