# A comprehensive study of LLM-based argument classification: from LLAMA through GPT-4o to Deepseek-R1

arXiv: https://arxiv.org/abs/2507.08621 · HTML: https://mlsanity.com/paper/2507.08621/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 11 Jul 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves](https://mlsanity.com/paper/2311.04205/) (2023)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks](https://mlsanity.com/paper/1802.05758/) (2018)
- [Classification and Clustering of Arguments with Contextualized Word Embeddings](https://mlsanity.com/paper/1906.09821/) (2019)
- [Fairness-guided Few-shot Prompting for Large Language Models](https://mlsanity.com/paper/2303.13217/) (2023)
- [Testing AI on language comprehension tasks reveals insensitivity to underlying meaning](https://mlsanity.com/paper/2302.12313/) (2023)
- [AMPERSAND: Argument Mining for PERSuAsive oNline Discussions](https://mlsanity.com/paper/2004.14677/) (2020)
- [Argument Mining with Structured SVMs and RNNs](https://mlsanity.com/paper/1704.06869/) (2017)
- [What is the Essence of a Claim? Cross-Domain Claim Identification](https://mlsanity.com/paper/1704.07203/) (2017)
- [Improving Retrieval Augmented Language Model with Self-Reasoning](https://mlsanity.com/paper/2407.19813/) (2024)
- [Transformer-Based Models for Automatic Identification of Argument Relations: A Cross-Domain Evaluation](https://mlsanity.com/paper/2011.13187/) (2020)
- [Self-Reflection Makes Large Language Models Safer, Less Biased, and Ideologically Neutral](https://mlsanity.com/paper/2406.10400/) (2024)
- [Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns](https://mlsanity.com/paper/2404.14812/) (2024)
- [A Hybrid Intelligence Method for Argument Mining](https://mlsanity.com/paper/2403.09713/) (2024)
- [Efficient argument classification with compact language models and ChatGPT-4 refinements](https://mlsanity.com/paper/2403.15473/) (2024)
- [Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion](https://mlsanity.com/paper/2405.11464/) (2024)
- [Fundamental Principles of Linguistic Structure are Not Represented by o3](https://mlsanity.com/paper/2502.10934/) (2025)

## Built on by (most cited first, 1 of 1)

- [ArgBench: Benchmarking LLMs on Computational Argumentation Tasks](https://mlsanity.com/paper/2604.17366/) (2026)

## Abstract (arXiv)

Argument mining (AM) is an interdisciplinary research field that integrates insights from logic, philosophy, linguistics, rhetoric, law, psychology, and computer science. It involves the automatic identification and extraction of argumentative components, such as premises and claims, and the detection of relationships between them, such as support, attack, or neutrality. Recently, the field has advanced significantly, especially with the advent of large language models (LLMs), which have enhanced the efficiency of analyzing and extracting argument semantics compared to traditional methods and other deep learning models. There are many benchmarks for testing and verifying the quality of LLM, but there is still a lack of research and results on the operation of these models in publicly available argument classification databases. This paper presents a study of a selection of LLM's, using diverse datasets such as Args.me and UKP. The models tested include versions of GPT, Llama, and DeepSeek, along with reasoning-enhanced variants incorporating the Chain-of-Thoughts algorithm. The results indicate that ChatGPT-4o outperforms the others in the argument classification benchmarks. In case of models incorporated with reasoning capabilities, the Deepseek-R1 shows its superiority. However, despite their superiority, GPT-4o and Deepseek-R1 still make errors. The most common errors are discussed for all models. To our knowledge, the presented work is the first broader analysis of the mentioned datasets using LLM and prompt algorithms. The work also shows some weaknesses of known prompt algorithms in argument analysis, while indicating directions for their improvement. The added value of the work is the in-depth analysis of the available argument datasets and the demonstration of their shortcomings.