# Improving Graph-Based Text Representations with Character and Word Level N-grams

arXiv: https://arxiv.org/abs/2210.05999 · HTML: https://mlsanity.com/paper/2210.05999/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks](https://mlsanity.com/paper/1508.00200/) (2015)
- [Heterogeneous Graph Neural Networks for Extractive Document Summarization](https://mlsanity.com/paper/2004.12393/) (2020)
- [Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints](https://mlsanity.com/paper/1603.08887/) (2016)
- [Graph-based Modeling of Online Communities for Fake News Detection](https://mlsanity.com/paper/2008.06274/) (2020)
- [HeteGCN: Heterogeneous Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2008.12842/) (2020)
- [Enhancing Scientific Papers Summarization with Citation Graph](https://mlsanity.com/paper/2104.03057/) (2021)
- [Knowledge-Aware Meta-learning for Low-Resource Text Classification](https://mlsanity.com/paper/2109.04707/) (2021)
- [Predicting Twitter User Socioeconomic Attributes with Network and Language Information](https://mlsanity.com/paper/1804.04095/) (2018)
- [Nowcasting the Stance of Social Media Users in a Sudden Vote: The Case of the Greek Referendum](https://mlsanity.com/paper/1808.08538/) (2018)
- [An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction](https://mlsanity.com/paper/2109.01238/) (2021)
- [Identifying and Characterizing Active Citizens who Refute Misinformation in Social Media](https://mlsanity.com/paper/2204.10080/) (2022)

## Built on by (most cited first, 2 of 2)

- [Rethinking Graph-Based Document Classification: Learning Data-Driven Structures Beyond Heuristic Approaches](https://mlsanity.com/paper/2508.00864/) (2025)
- [ModTGCN: Modularity-aware Graph Neural Networks for Text Classification](https://mlsanity.com/paper/2606.23694/) (2026)

## Abstract (arXiv)

Graph-based text representation focuses on how text documents are represented as graphs for exploiting dependency information between tokens and documents within a corpus. Despite the increasing interest in graph representation learning, there is limited research in exploring new ways for graph-based text representation, which is important in downstream natural language processing tasks. In this paper, we first propose a new heterogeneous word-character text graph that combines word and character n-gram nodes together with document nodes, allowing us to better learn dependencies among these entities. Additionally, we propose two new graph-based neural models, WCTextGCN and WCTextGAT, for modeling our proposed text graph. Extensive experiments in text classification and automatic text summarization benchmarks demonstrate that our proposed models consistently outperform competitive baselines and state-of-the-art graph-based models.