# Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining

arXiv: https://arxiv.org/abs/2312.00874 · HTML: https://mlsanity.com/paper/2312.00874/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Stance Detection with Bidirectional Conditional Encoding](https://mlsanity.com/paper/1606.05464/) (2016)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [From Arguments to Key Points: Towards Automatic Argument Summarization](https://mlsanity.com/paper/2005.01619/) (2020)
- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [Neural Network-Based Abstract Generation for Opinions and Arguments](https://mlsanity.com/paper/1606.02785/) (2016)
- [Aspect-Controlled Neural Argument Generation](https://mlsanity.com/paper/2005.00084/) (2020)
- [Neural End-to-End Learning for Computational Argumentation Mining](https://mlsanity.com/paper/1704.06104/) (2017)
- [Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis](https://mlsanity.com/paper/2010.05369/) (2020)
- [Overview of the 2021 Key Point Analysis Shared Task](https://mlsanity.com/paper/2110.10577/) (2021)
- [IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks](https://mlsanity.com/paper/2203.12257/) (2022)
- [The Moral Debater: A Study on the Computational Generation of Morally Framed Arguments](https://mlsanity.com/paper/2203.14563/) (2022)
- [Key Point Analysis via Contrastive Learning and Extractive Argument Summarization](https://mlsanity.com/paper/2109.15086/) (2021)
- [The workweek is the best time to start a family -- A Study of GPT-2 Based Claim Generation](https://mlsanity.com/paper/2010.06185/) (2020)
- [Team Enigma at ArgMining-EMNLP 2021: Leveraging Pre-trained Language Models for Key Point Matching](https://mlsanity.com/paper/2110.12370/) (2021)

## Built on by (most cited first, 2 of 2)

- [Overview of AI-Debater 2023: The Challenges of Argument Generation Tasks](https://mlsanity.com/paper/2407.14829/) (2024)
- [Overview of the CAIL 2023 Argument Mining Track](https://mlsanity.com/paper/2406.14503/) (2024)

## Abstract (arXiv)

The knowledge graph is a structure to store and represent knowledge, and recent studies have discussed its capability to assist language models for various applications. Some variations of knowledge graphs aim to record arguments and their relations for computational argumentation tasks. However, many must simplify semantic types to fit specific schemas, thus losing flexibility and expression ability. In this paper, we propose the Hierarchical Argumentation Graph (Hi-ArG), a new structure to organize arguments. We also introduce two approaches to exploit Hi-ArG, including a text-graph multi-modal model GreaseArG and a new pre-training framework augmented with graph information. Experiments on two argumentation tasks have shown that after further pre-training and fine-tuning, GreaseArG supersedes same-scale language models on these tasks, while incorporating graph information during further pre-training can also improve the performance of vanilla language models. Code for this paper is available at https://github.com/ljcleo/Hi-ArG .