# Exploring the Potential of Large Language Models in Computational Argumentation

arXiv: https://arxiv.org/abs/2311.09022 · HTML: https://mlsanity.com/paper/2311.09022/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 95 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 30 of 51)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://mlsanity.com/paper/2305.12744/) (2023)
- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization](https://mlsanity.com/paper/2302.08081/) (2023)
- [A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability](https://mlsanity.com/paper/2303.13547/) (2023)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study](https://mlsanity.com/paper/2304.04339/) (2023)
- [Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models](https://mlsanity.com/paper/2305.01219/) (2023)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization](https://mlsanity.com/paper/2305.13091/) (2023)
- [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://mlsanity.com/paper/2304.04193/) (2023)
- [The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants](https://mlsanity.com/paper/1708.01425/) (2017)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [ChatGPT: A Meta-Analysis after 2.5 Months](https://mlsanity.com/paper/2302.13795/) (2023)

## Built on by (most cited first, 30 of 42)

- [Argumentative Large Language Models for Explainable and Contestable Claim Verification](https://mlsanity.com/paper/2405.02079/) (2024)
- [Are Large Language Models Reliable Argument Quality Annotators?](https://mlsanity.com/paper/2404.09696/) (2024)
- [Can Large Language Models perform Relation-based Argument Mining?](https://mlsanity.com/paper/2402.11243/) (2024)
- [Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM](https://mlsanity.com/paper/2403.08010/) (2024)
- [RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery](https://mlsanity.com/paper/2503.00751/) (2025)
- [Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive Debate](https://mlsanity.com/paper/2408.04472/) (2024)
- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes](https://mlsanity.com/paper/2403.06294/) (2024)
- [Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey](https://mlsanity.com/paper/2503.01513/) (2025)
- [A Logical Fallacy-Informed Framework for Argument Generation](https://mlsanity.com/paper/2408.03618/) (2024)
- [Computational Argumentation-based Chatbots: a Survey](https://mlsanity.com/paper/2401.03454/) (2024)
- [Semantic Change Characterization with LLMs using Rhetorics](https://mlsanity.com/paper/2407.16624/) (2024)
- [The Thin Line Between Comprehension and Persuasion in LLMs](https://mlsanity.com/paper/2507.01936/) (2025)
- ["Let's Argue Both Sides": Argument Generation Can Force Small Models to Utilize Previously Inaccessible Reasoning Capabilities](https://mlsanity.com/paper/2410.12997/) (2024)
- [ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentation](https://mlsanity.com/paper/2505.22076/) (2025)
- [LLMs for Argument Mining: Detection, Extraction, and Relationship Classification of pre-defined Arguments in Online Comments](https://mlsanity.com/paper/2505.22956/) (2025)
- [Critical Questions Generation: Motivation and Challenges](https://mlsanity.com/paper/2410.14335/) (2024)
- [Enhancing Conflict Resolution in Language Models via Abstract Argumentation](https://mlsanity.com/paper/2412.16725/) (2024)
- [Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences](https://mlsanity.com/paper/2507.08440/) (2025)
- [AMELIA: A Family of Multi-task End-to-end Language Models for Argumentation](https://mlsanity.com/paper/2508.17926/) (2025)
- [WIBA: What Is Being Argued? A Comprehensive Approach to Argument Mining](https://mlsanity.com/paper/2405.00828/) (2024)
- [Towards Comprehensive Argument Analysis in Education: Dataset, Tasks, and Method](https://mlsanity.com/paper/2505.12028/) (2025)
- [Stakeholder Suite: A Unified AI Framework for Mapping Actors, Topics and Arguments in Public Debates](https://mlsanity.com/paper/2512.17347/) (2025)
- [On Theoretically-Driven LLM Agents for Multi-Dimensional Discourse Analysis](https://mlsanity.com/paper/2602.13713/) (2026)
- [Compact Prompting in Instruction-tuned LLMs for Joint Argumentative Component Detection](https://mlsanity.com/paper/2603.03095/) (2026)
- [ArgBench: Benchmarking LLMs on Computational Argumentation Tasks](https://mlsanity.com/paper/2604.17366/) (2026)
- [Beyond Reproduction: A Paired-Task Framework for Assessing LLM Comprehension and Creativity in Literary Translation](https://mlsanity.com/paper/2604.18169/) (2026)
- [Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts](https://mlsanity.com/paper/2409.02519/) (2024)
- [CLEAR: A Comprehensive Linguistic Evaluation of Argument Rewriting by Large Language Models](https://mlsanity.com/paper/2509.15027/) (2025)
- [The GDN-CC Dataset: Automatic Corpus Clarification for AI-enhanced Democratic Citizen Consultations](https://mlsanity.com/paper/2601.14944/) (2026)

## Abstract (arXiv)

Computational argumentation has become an essential tool in various domains, including law, public policy, and artificial intelligence. It is an emerging research field in natural language processing that attracts increasing attention. Research on computational argumentation mainly involves two types of tasks: argument mining and argument generation. As large language models (LLMs) have demonstrated impressive capabilities in understanding context and generating natural language, it is worthwhile to evaluate the performance of LLMs on diverse computational argumentation tasks. This work aims to embark on an assessment of LLMs, such as ChatGPT, Flan models, and LLaMA2 models, in both zero-shot and few-shot settings. We organize existing tasks into six main categories and standardize the format of fourteen openly available datasets. In addition, we present a new benchmark dataset on counter speech generation that aims to holistically evaluate the end-to-end performance of LLMs on argument mining and argument generation. Extensive experiments show that LLMs exhibit commendable performance across most of the datasets, demonstrating their capabilities in the field of argumentation. Our analysis offers valuable suggestions for evaluating computational argumentation and its integration with LLMs in future research endeavors.