# Rethinking Graph-Based Document Classification: Learning Data-Driven Structures Beyond Heuristic Approaches

arXiv: https://arxiv.org/abs/2508.00864 · HTML: https://mlsanity.com/paper/2508.00864/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 18 Jul 2025.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection](https://mlsanity.com/paper/2306.04637/) (2023)
- [Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks](https://mlsanity.com/paper/2004.13826/) (2020)
- [Text Level Graph Neural Network for Text Classification](https://mlsanity.com/paper/1910.02356/) (2019)
- [Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network](https://mlsanity.com/paper/1906.04684/) (2019)
- [Replacing softmax with ReLU in Vision Transformers](https://mlsanity.com/paper/2309.08586/) (2023)
- [Efficient Classification of Long Documents Using Transformers](https://mlsanity.com/paper/2203.11258/) (2022)
- [Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks](https://mlsanity.com/paper/2010.06253/) (2020)
- [LSG Attention: Extrapolation of pretrained Transformers to long sequences](https://mlsanity.com/paper/2210.15497/) (2022)
- [Recurrent Attention Networks for Long-text Modeling](https://mlsanity.com/paper/2306.06843/) (2023)
- [Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification](https://mlsanity.com/paper/2406.01283/) (2024)
- [Contrastive Document Representation Learning with Graph Attention Networks](https://mlsanity.com/paper/2110.10778/) (2021)
- [Contrastive Multi-graph Learning with Neighbor Hierarchical Sifting for Semi-supervised Text Classification](https://mlsanity.com/paper/2411.16787/) (2024)
- [GraphLSS: Integrating Lexical, Structural, and Semantic Features for Long Document Extractive Summarization](https://mlsanity.com/paper/2410.21315/) (2024)
- [Improving Graph-Based Text Representations with Character and Word Level N-grams](https://mlsanity.com/paper/2210.05999/) (2022)
- [A prototype-based model for set classification](https://mlsanity.com/paper/2408.13720/) (2024)

## Built on by (most cited first, 1 of 1)

- [From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization](https://mlsanity.com/paper/2603.00021/) (2026)

## Abstract (arXiv)

In document classification, graph-based models effectively capture document structure, overcoming sequence length limitations and enhancing contextual understanding. However, most existing graph document representations rely on heuristics, domain-specific rules, or expert knowledge. Unlike previous approaches, we propose a method to learn data-driven graph structures, eliminating the need for manual design and reducing domain dependence. Our approach constructs homogeneous weighted graphs with sentences as nodes, while edges are learned via a self-attention model that identifies dependencies between sentence pairs. A statistical filtering strategy aims to retain only strongly correlated sentences, improving graph quality while reducing the graph size. Experiments on three document classification datasets demonstrate that learned graphs consistently outperform heuristic-based graphs, achieving higher accuracy and F1 score. Furthermore, our study demonstrates the effectiveness of the statistical filtering in improving classification robustness. These results highlight the potential of automatic graph generation over traditional heuristic approaches and open new directions for broader applications in NLP.