# Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?

arXiv: https://arxiv.org/abs/2305.14578 · HTML: https://mlsanity.com/paper/2305.14578/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks](https://mlsanity.com/paper/2004.13826/) (2020)
- [Text Level Graph Neural Network for Text Classification](https://mlsanity.com/paper/1910.02356/) (2019)
- [Tensor Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2001.05313/) (2020)
- [GraphIE: A Graph-Based Framework for Information Extraction](https://mlsanity.com/paper/1810.13083/) (2018)
- [Message Passing Attention Networks for Document Understanding](https://mlsanity.com/paper/1908.06267/) (2019)
- [Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis](https://mlsanity.com/paper/2109.08256/) (2021)
- [Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP](https://mlsanity.com/paper/2109.03777/) (2021)

## Abstract (arXiv)

Given the success of Graph Neural Networks (GNNs) for structure-aware machine learning, many studies have explored their use for text classification, but mostly in specific domains with limited data characteristics. Moreover, some strategies prior to GNNs relied on graph mining and classical machine learning, making it difficult to assess their effectiveness in modern settings. This work extensively investigates graph representation methods for text classification, identifying practical implications and open challenges. We compare different graph construction schemes using a variety of GNN architectures and setups across five datasets, encompassing short and long documents as well as unbalanced scenarios in diverse domains. Two Transformer-based large language models are also included to complement the study. The results show that i) although the effectiveness of graphs depends on the textual input features and domain, simple graph constructions perform better the longer the documents are, ii) graph representations are especially beneficial for longer documents, outperforming Transformer-based models, iii) graph methods are particularly efficient at solving the task.