# MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction

arXiv: https://arxiv.org/abs/2106.12940 · HTML: https://mlsanity.com/paper/2106.12940/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks](https://mlsanity.com/paper/2004.07464/) (2020)
- [Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solution](https://mlsanity.com/paper/2102.06732/) (2021)
- [BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding](https://mlsanity.com/paper/1909.04948/) (2019)
- [TRIE: End-to-End Text Reading and Information Extraction for Document Understanding](https://mlsanity.com/paper/2005.13118/) (2020)
- [GraphIE: A Graph-Based Framework for Information Extraction](https://mlsanity.com/paper/1810.13083/) (2018)
- [EATEN: Entity-aware Attention for Single Shot Visual Text Extraction](https://mlsanity.com/paper/1909.09380/) (2019)
- [A Neural Model for Text Localization, Transcription and Named Entity Recognition in Full Pages](https://mlsanity.com/paper/1912.10016/) (2019)
- [One-shot Text Field Labeling using Attention and Belief Propagation for Structure Information Extraction](https://mlsanity.com/paper/2009.04153/) (2020)

## Built on by (most cited first, 18 of 18)

- [LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding](https://mlsanity.com/paper/2202.13669/) (2022)
- [StrucTexT: Structured Text Understanding with Multi-Modal Transformers](https://mlsanity.com/paper/2108.02923/) (2021)
- [OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition](https://mlsanity.com/paper/2403.19128/) (2024)
- [Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution](https://mlsanity.com/paper/2305.07498/) (2023)
- [GeoLayoutLM: Geometric Pre-training for Visual Information Extraction](https://mlsanity.com/paper/2304.10759/) (2023)
- [WebFormer: The Web-page Transformer for Structure Information Extraction](https://mlsanity.com/paper/2202.00217/) (2022)
- [OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models](https://mlsanity.com/paper/2502.16161/) (2025)
- [Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding](https://mlsanity.com/paper/2206.13155/) (2022)
- [Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding](https://mlsanity.com/paper/2409.19672/) (2024)
- [GMN: Generative Multi-modal Network for Practical Document Information Extraction](https://mlsanity.com/paper/2207.04713/) (2022)
- [TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2207.06744/) (2022)
- [Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding](https://mlsanity.com/paper/2305.11392/) (2023)
- [SAIL: Sample-Centric In-Context Learning for Document Information Extraction](https://mlsanity.com/paper/2412.17092/) (2024)
- [Handling big tabular data of ICT supply chains: a multi-task, machine-interpretable approach](https://mlsanity.com/paper/2208.06031/) (2022)
- [Global Structure Knowledge-Guided Relation Extraction Method for Visually-Rich Document](https://mlsanity.com/paper/2305.13850/) (2023)
- [Vision-Enhanced Semantic Entity Recognition in Document Images via Visually-Asymmetric Consistency Learning](https://mlsanity.com/paper/2310.14785/) (2023)
- [Towards Human-Like Machine Comprehension: Few-Shot Relational Learning in Visually-Rich Documents](https://mlsanity.com/paper/2403.15765/) (2024)
- [Visual Information Extraction from Documents via Classification-Guided Large Vision-Language Models](https://mlsanity.com/paper/2607.22723/) (2026)

## Abstract (arXiv)

Visual Information Extraction (VIE) task aims to extract key information from multifarious document images (e.g., invoices and purchase receipts). Most previous methods treat the VIE task simply as a sequence labeling problem or classification problem, which requires models to carefully identify each kind of semantics by introducing multimodal features, such as font, color, layout. But simply introducing multimodal features couldn't work well when faced with numeric semantic categories or some ambiguous texts. To address this issue, in this paper we propose a novel key-value matching model based on a graph neural network for VIE (MatchVIE). Through key-value matching based on relevancy evaluation, the proposed MatchVIE can bypass the recognitions to various semantics, and simply focuses on the strong relevancy between entities. Besides, we introduce a simple but effective operation, Num2Vec, to tackle the instability of encoded values, which helps model converge more smoothly. Comprehensive experiments demonstrate that the proposed MatchVIE can significantly outperform previous methods. Notably, to the best of our knowledge, MatchVIE may be the first attempt to tackle the VIE task by modeling the relevancy between keys and values and it is a good complement to the existing methods.