# Visual FUDGE: Form Understanding via Dynamic Graph Editing

arXiv: https://arxiv.org/abs/2105.08194 · HTML: https://mlsanity.com/paper/2105.08194/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding](https://mlsanity.com/paper/2010.11685/) (2020)
- [Attend, Copy, Parse -- End-to-end information extraction from documents](https://mlsanity.com/paper/1812.07248/) (2018)
- [Deep Visual Template-Free Form Parsing](https://mlsanity.com/paper/1909.02576/) (2019)

## Built on by (most cited first, 15 of 15)

- [End-to-end Document Recognition and Understanding with Dessurt](https://mlsanity.com/paper/2203.16618/) (2022)
- [PDFVQA: A New Dataset for Real-World VQA on PDF Documents](https://mlsanity.com/paper/2304.06447/) (2023)
- [Doc2Graph: a Task Agnostic Document Understanding Framework based on Graph Neural Networks](https://mlsanity.com/paper/2208.11168/) (2022)
- [A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images](https://mlsanity.com/paper/2304.07957/) (2023)
- [BuDDIE: A Business Document Dataset for Multi-task Information Extraction](https://mlsanity.com/paper/2404.04003/) (2024)
- [DocGraphLM: Documental Graph Language Model for Information Extraction](https://mlsanity.com/paper/2401.02823/) (2024)
- [CommonForms: A Large, Diverse Dataset for Form Field Detection](https://mlsanity.com/paper/2509.16506/) (2025)
- [TreeForm: End-to-end Annotation and Evaluation for Form Document Parsing](https://mlsanity.com/paper/2402.05282/) (2024)
- [GeoContrastNet: Contrastive Key-Value Edge Learning for Language-Agnostic Document Understanding](https://mlsanity.com/paper/2405.03104/) (2024)
- [Unimodal and Multimodal Representation Training for Relation Extraction](https://mlsanity.com/paper/2211.06168/) (2022)
- [PARAGRAPH2GRAPH: A GNN-based framework for layout paragraph analysis](https://mlsanity.com/paper/2304.11810/) (2023)
- [PEneo: Unifying Line Extraction, Line Grouping, and Entity Linking for End-to-end Document Pair Extraction](https://mlsanity.com/paper/2401.03472/) (2024)
- [Text Classification Models for Form Entity Linking](https://mlsanity.com/paper/2112.07443/) (2021)
- [Language Independent Neuro-Symbolic Semantic Parsing for Form Understanding](https://mlsanity.com/paper/2305.04460/) (2023)
- [Improving Document Image Understanding with Reinforcement Finetuning](https://mlsanity.com/paper/2209.12561/) (2022)

## Abstract (arXiv)

We address the problem of form understanding: finding text entities and the relationships/links between them in form images. The proposed FUDGE model formulates this problem on a graph of text elements (the vertices) and uses a Graph Convolutional Network to predict changes to the graph. The initial vertices are detected text lines and do not necessarily correspond to the final text entities, which can span multiple lines. Also, initial edges contain many false-positive relationships. FUDGE edits the graph structure by combining text segments (graph vertices) and pruning edges in an iterative fashion to obtain the final text entities and relationships. While recent work in this area has focused on leveraging large-scale pre-trained Language Models (LM), FUDGE achieves almost the same level of entity linking performance on the FUNSD dataset by learning only visual features from the (small) provided training set. FUDGE can be applied on forms where text recognition is difficult (e.g. degraded or historical forms) and on forms in resource-poor languages where pre-training such LMs is challenging. FUDGE is state-of-the-art on the historical NAF dataset.