# Graph Neural Networks on Discriminative Graphs of Words

arXiv: https://arxiv.org/abs/2410.20469 · HTML: https://mlsanity.com/paper/2410.20469/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 27 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Text Classification Algorithms: A Survey](https://mlsanity.com/paper/1904.08067/) (2019)
- [BERT for Coreference Resolution: Baselines and Analysis](https://mlsanity.com/paper/1908.09091/) (2019)
- [PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks](https://mlsanity.com/paper/1508.00200/) (2015)
- [Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks](https://mlsanity.com/paper/2004.13826/) (2020)
- [Be More with Less: Hypergraph Attention Networks for Inductive Text Classification](https://mlsanity.com/paper/2011.00387/) (2020)
- [Tensor Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2001.05313/) (2020)
- [Message Passing Attention Networks for Document Understanding](https://mlsanity.com/paper/1908.06267/) (2019)
- [HeteGCN: Heterogeneous Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2008.12842/) (2020)
- [Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP](https://mlsanity.com/paper/2109.03777/) (2021)
- [Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification](https://mlsanity.com/paper/2112.06386/) (2021)
- [InducT-GCN: Inductive Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2206.00265/) (2022)
- [Weighted Spectral Embedding of Graphs](https://mlsanity.com/paper/1809.11115/) (2018)
- [Simple Graph Convolutional Networks](https://mlsanity.com/paper/2106.05809/) (2021)

## Abstract (arXiv)

In light of the recent success of Graph Neural Networks (GNNs) and their ability to perform inference on complex data structures, many studies apply GNNs to the task of text classification. In most previous methods, a heterogeneous graph, containing both word and document nodes, is constructed using the entire corpus and a GNN is used to classify document nodes. In this work, we explore a new Discriminative Graph of Words Graph Neural Network (DGoW-GNN) approach encapsulating both a novel discriminative graph construction and model to classify text. In our graph construction, containing only word nodes and no document nodes, we split the training corpus into disconnected subgraphs according to their labels and weight edges by the pointwise mutual information of the represented words. Our graph construction, for which we provide theoretical motivation, allows us to reformulate the task of text classification as the task of walk classification. We also propose a new model for the graph-based classification of text, which combines a GNN and a sequence model. We evaluate our approach on seven benchmark datasets and find that it is outperformed by several state-of-the-art baseline models. We analyse reasons for this performance difference and hypothesise under which conditions it is likely to change.