# GVdoc: Graph-based Visual Document Classification

arXiv: https://arxiv.org/abs/2305.17219 · HTML: https://mlsanity.com/paper/2305.17219/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval](https://mlsanity.com/paper/1502.07058/) (2015)
- [DiT: Self-supervised Pre-training for Document Image Transformer](https://mlsanity.com/paper/2203.02378/) (2022)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [SelfDoc: Self-Supervised Document Representation Learning](https://mlsanity.com/paper/2106.03331/) (2021)
- [LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding](https://mlsanity.com/paper/2202.13669/) (2022)
- [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://mlsanity.com/paper/2108.04539/) (2021)
- [StructuralLM: Structural Pre-training for Form Understanding](https://mlsanity.com/paper/2105.11210/) (2021)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [Deep Relational Reasoning Graph Network for Arbitrary Shape Text Detection](https://mlsanity.com/paper/2003.07493/) (2020)
- [PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks](https://mlsanity.com/paper/2004.07464/) (2020)
- [ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding](https://mlsanity.com/paper/2210.06155/) (2022)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [LAMBERT: Layout-Aware (Language) Modeling for information extraction](https://mlsanity.com/paper/2002.08087/) (2020)
- [ETC: Encoding Long and Structured Inputs in Transformers](https://mlsanity.com/paper/2004.08483/) (2020)
- [ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction](https://mlsanity.com/paper/2106.10786/) (2021)
- [Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1801.09321/) (2018)
- [Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models](https://mlsanity.com/paper/2005.11017/) (2020)
- [One-shot Text Field Labeling using Attention and Belief Propagation for Structure Information Extraction](https://mlsanity.com/paper/2009.04153/) (2020)
- [Post-OCR Paragraph Recognition by Graph Convolutional Networks](https://mlsanity.com/paper/2101.12741/) (2021)
- [Evaluating Out-of-Distribution Performance on Document Image Classifiers](https://mlsanity.com/paper/2210.07448/) (2022)
- [Position Masking for Improved Layout-Aware Document Understanding](https://mlsanity.com/paper/2109.00442/) (2021)
- [Exploring Out-of-Distribution Generalization in Text Classifiers Trained on Tobacco-3482 and RVL-CDIP](https://mlsanity.com/paper/2108.02684/) (2021)

## Built on by (most cited first, 1 of 1)

- [Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding](https://mlsanity.com/paper/2409.19672/) (2024)

## Abstract (arXiv)

The robustness of a model for real-world deployment is decided by how well it performs on unseen data and distinguishes between in-domain and out-of-domain samples. Visual document classifiers have shown impressive performance on in-distribution test sets. However, they tend to have a hard time correctly classifying and differentiating out-of-distribution examples. Image-based classifiers lack the text component, whereas multi-modality transformer-based models face the token serialization problem in visual documents due to their diverse layouts. They also require a lot of computing power during inference, making them impractical for many real-world applications. We propose, GVdoc, a graph-based document classification model that addresses both of these challenges. Our approach generates a document graph based on its layout, and then trains a graph neural network to learn node and graph embeddings. Through experiments, we show that our model, even with fewer parameters, outperforms state-of-the-art models on out-of-distribution data while retaining comparable performance on the in-distribution test set.