# VisualWordGrid: Information Extraction From Scanned Documents Using A Multimodal Approach

arXiv: https://arxiv.org/abs/2010.02358 · HTML: https://mlsanity.com/paper/2010.02358/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents](https://mlsanity.com/paper/2105.11672/) (2021)
- [DocParser: End-to-end OCR-free Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2304.12484/) (2023)
- [LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding](https://mlsanity.com/paper/2104.08405/) (2021)
- [Exploring the Capabilities of Large Multimodal Models on Dense Text](https://mlsanity.com/paper/2405.06706/) (2024)
- [Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Review](https://mlsanity.com/paper/2408.06345/) (2024)
- [Position Masking for Improved Layout-Aware Document Understanding](https://mlsanity.com/paper/2109.00442/) (2021)
- [Information Extraction from Visually Rich Documents with Font Style Embeddings](https://mlsanity.com/paper/2111.04045/) (2021)
- [ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents](https://mlsanity.com/paper/2409.15004/) (2024)

## Abstract (arXiv)

We introduce a novel approach for scanned document representation to perform field extraction. It allows the simultaneous encoding of the textual, visual and layout information in a 3-axis tensor used as an input to a segmentation model. We improve the recent Chargrid and Wordgrid citechargrid models in several ways, first by taking into account the visual modality, then by boosting its robustness in regards to small datasets while keeping the inference time low. Our approach is tested on public and private document-image datasets, showing higher performances compared to the recent state-of-the-art methods.