# GraphIE: A Graph-Based Framework for Information Extraction

arXiv: https://arxiv.org/abs/1810.13083 · HTML: https://mlsanity.com/paper/1810.13083/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 30 of 45)

- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding](https://mlsanity.com/paper/2202.13669/) (2022)
- [StrucTexT: Structured Text Understanding with Multi-Modal Transformers](https://mlsanity.com/paper/2108.02923/) (2021)
- [Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solution](https://mlsanity.com/paper/2102.06732/) (2021)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution](https://mlsanity.com/paper/2305.07498/) (2023)
- [GeoLayoutLM: Geometric Pre-training for Visual Information Extraction](https://mlsanity.com/paper/2304.10759/) (2023)
- [ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents](https://mlsanity.com/paper/2105.11672/) (2021)
- [Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering](https://mlsanity.com/paper/2306.00526/) (2023)
- [ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction](https://mlsanity.com/paper/2106.10786/) (2021)
- [MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction](https://mlsanity.com/paper/2106.12940/) (2021)
- [Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding](https://mlsanity.com/paper/2206.13155/) (2022)
- [Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentration](https://mlsanity.com/paper/2309.01131/) (2023)
- [Cost-effective End-to-end Information Extraction for Semi-structured Document Images](https://mlsanity.com/paper/2104.08041/) (2021)
- [Towards reducing hallucination in extracting information from financial reports using Large Language Models](https://mlsanity.com/paper/2310.10760/) (2023)
- [ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding](https://mlsanity.com/paper/2209.08569/) (2022)
- [Universal Graph Continual Learning](https://mlsanity.com/paper/2308.13982/) (2023)
- [Tag, Copy or Predict: A Unified Weakly-Supervised Learning Framework for Visual Information Extraction using Sequences](https://mlsanity.com/paper/2106.10681/) (2021)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [A Span Extraction Approach for Information Extraction on Visually-Rich Documents](https://mlsanity.com/paper/2106.00978/) (2021)
- [TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2207.06744/) (2022)
- [DocGraphLM: Documental Graph Language Model for Information Extraction](https://mlsanity.com/paper/2401.02823/) (2024)
- [SAIL: Sample-Centric In-Context Learning for Document Information Extraction](https://mlsanity.com/paper/2412.17092/) (2024)
- [Using Neighborhood Context to Improve Information Extraction from Visual Documents Captured on Mobile Phones](https://mlsanity.com/paper/2108.10395/) (2021)
- [Larger-Context Tagging: When and Why Does It Work?](https://mlsanity.com/paper/2104.04434/) (2021)
- [A New Task and Dataset on Detecting Attacks on Human Rights Defenders](https://mlsanity.com/paper/2306.17695/) (2023)
- [Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding](https://mlsanity.com/paper/2402.18262/) (2024)
- [One-shot Key Information Extraction from Document with Deep Partial Graph Matching](https://mlsanity.com/paper/2109.13967/) (2021)
- [Summarization with Graphical Elements](https://mlsanity.com/paper/2204.07551/) (2022)
- [HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System](https://mlsanity.com/paper/2206.02628/) (2022)

## Abstract (arXiv)

Most modern Information Extraction (IE) systems are implemented as sequential taggers and only model local dependencies. Non-local and non-sequential context is, however, a valuable source of information to improve predictions. In this paper, we introduce GraphIE, a framework that operates over a graph representing a broad set of dependencies between textual units (i.e. words or sentences). The algorithm propagates information between connected nodes through graph convolutions, generating a richer representation that can be exploited to improve word-level predictions. Evaluation on three different tasks --- namely textual, social media and visual information extraction --- shows that GraphIE consistently outperforms the state-of-the-art sequence tagging model by a significant margin.