# BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding

arXiv: https://arxiv.org/abs/1909.04948 · HTML: https://mlsanity.com/paper/1909.04948/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2019.

## Built on by (most cited first, 30 of 63)

- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding](https://mlsanity.com/paper/2202.13669/) (2022)
- [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://mlsanity.com/paper/2108.04539/) (2021)
- [StrucTexT: Structured Text Understanding with Multi-Modal Transformers](https://mlsanity.com/paper/2108.02923/) (2021)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [Vision Grid Transformer for Document Layout Analysis](https://mlsanity.com/paper/2308.14978/) (2023)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [LMDX: Language Model-based Document Information Extraction and Localization](https://mlsanity.com/paper/2309.10952/) (2023)
- [ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents](https://mlsanity.com/paper/2105.11672/) (2021)
- [Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering](https://mlsanity.com/paper/2306.00526/) (2023)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction](https://mlsanity.com/paper/2305.02549/) (2023)
- [DocParser: End-to-end OCR-free Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2304.12484/) (2023)
- [MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction](https://mlsanity.com/paper/2106.12940/) (2021)
- [Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding](https://mlsanity.com/paper/2206.13155/) (2022)
- [DocTr: Document Transformer for Structured Information Extraction in Documents](https://mlsanity.com/paper/2307.07929/) (2023)
- [Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentration](https://mlsanity.com/paper/2309.01131/) (2023)
- [Form2Seq : A Framework for Higher-Order Form Structure Extraction](https://mlsanity.com/paper/2107.04419/) (2021)
- [LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding](https://mlsanity.com/paper/2104.08405/) (2021)
- [Cost-effective End-to-end Information Extraction for Semi-structured Document Images](https://mlsanity.com/paper/2104.08041/) (2021)
- [Business Document Information Extraction: Towards Practical Benchmarks](https://mlsanity.com/paper/2206.11229/) (2022)
- [ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding](https://mlsanity.com/paper/2209.08569/) (2022)
- [Information Extraction from Visually Rich Documents using LLM-based Organization of Documents into Independent Textual Segments](https://mlsanity.com/paper/2505.13535/) (2025)
- [BuDDIE: A Business Document Dataset for Multi-task Information Extraction](https://mlsanity.com/paper/2404.04003/) (2024)
- [A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends](https://mlsanity.com/paper/2507.09861/) (2025)
- [Key Information Extraction From Documents: Evaluation And Generator](https://mlsanity.com/paper/2106.14624/) (2021)

## Abstract (arXiv)

For understanding generic documents, information like font sizes, column layout, and generally the positioning of words may carry semantic information that is crucial for solving a downstream document intelligence task. Our novel BERTgrid, which is based on Chargrid by Katti et al. (2018), represents a document as a grid of contextualized word piece embedding vectors, thereby making its spatial structure and semantics accessible to the processing neural network. The contextualized embedding vectors are retrieved from a BERT language model. We use BERTgrid in combination with a fully convolutional network on a semantic instance segmentation task for extracting fields from invoices. We demonstrate its performance on tabulated line item and document header field extraction.