mlsanityData to 7 Oct 2026

BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding

arXiv 1909.04948 ↗First posted 11 Sep 2019cs.CL cs.CV cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

For understanding generic documents, information like font sizes, column layout, and generally the positioning of words may carry semantic information that is crucial for solving a downstream document intelligence task. Our novel BERTgrid, which is based on Chargrid by Katti et al. (2018), represents a document as a grid of contextualized word piece embedding vectors, thereby making its spatial structure and semantics accessible to the processing neural network. The contextualized embedding vectors are retrieved from a BERT language model. We use BERTgrid in combination with a fully convolutional network on a semantic instance segmentation task for extracting fields from invoices. We demonstrate its performance on tabulated line item and document header field extraction.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 10; 2023: 16; 2024: 11; 2025: 5; 2026: 2.

63 papers.

Sort:

By area:

  1. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  2. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  3. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  4. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  5. 2021StrucTexT: Structured Text Understanding with Multi-Modal Transformerscited by 90
  6. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  7. 2022DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognitioncited by 62
  8. 2021Document AI: Benchmarks, Models and Applicationscited by 61
  9. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  10. 2023Vision Grid Transformer for Document Layout Analysiscited by 45
  11. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
  12. 2023LMDX: Language Model-based Document Information Extraction and Localizationcited by 37
38 more
  1. 2021ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documentscited by 29
  2. 2023Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answeringcited by 24
  3. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  4. 2023FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extractioncited by 20
  5. 2023DocParser: End-to-end OCR-free Information Extraction from Visually Rich Documentscited by 19
  6. 2021MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extractioncited by 18
  7. 2022Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understandingcited by 18
  8. 2023DocTr: Document Transformer for Structured Information Extraction in Documentscited by 18
  9. 2023Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentrationcited by 18
  10. 2021Form2Seq : A Framework for Higher-Order Form Structure Extractioncited by 16
  11. 2021LAMPRET: Layout-Aware Multimodal PreTraining for Document Understandingcited by 15
  12. 2021Cost-effective End-to-end Information Extraction for Semi-structured Document Imagescited by 14
  13. 2022Business Document Information Extraction: Towards Practical Benchmarkscited by 11
  14. 2022ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understandingcited by 9
  15. 2025Information Extraction from Visually Rich Documents using LLM-based Organization of Documents into Independent Textual Segmentscited by 9
  16. 2024BuDDIE: A Business Document Dataset for Multi-task Information Extractioncited by 8
  17. 2025A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trendscited by 8
  18. 2021Key Information Extraction From Documents: Evaluation And Generatorcited by 7
  19. 2021Skim-Attention: Learning to Focus via Document Layoutcited by 7
  20. 2022QueryForm: A Simple Zero-shot Form Entity Query Frameworkcited by 7
  21. 2022TRIE++: Towards End-to-End Information Extraction from Visually Rich Documentscited by 6
  22. 2024Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Reviewcited by 6
  23. 2024SAIL: Sample-Centric In-Context Learning for Document Information Extractioncited by 6
  24. 2023Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphscited by 5
  25. 2024XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parsercited by 5
  26. 2021DocReader: Bounding-Box Free Training of a Document Information Extraction Modelcited by 4
  27. 2021Field Extraction from Forms with Unlabeled Datacited by 4
  28. 2024ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-trainingcited by 4
  29. 2024Arabic-Nougat: Fine-Tuning Vision Transformers for Arabic OCR and Markdown Extractioncited by 4
  30. 2021DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extractioncited by 3
  31. 2021Efficient Document Image Classification Using Region-Based Graph Neural Networkcited by 3
  32. 2021Position Masking for Improved Layout-Aware Document Understandingcited by 3
  33. 2021One-shot Key Information Extraction from Document with Deep Partial Graph Matchingcited by 3
  34. 2023Improving Information Extraction on Business Documents with Specific Pre-Training Taskscited by 3
  35. 2025UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysiscited by 3
  36. 2025SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understandingcited by 3
  37. 2022Data-Efficient Information Extraction from Form-Like Documentscited by 2
  38. 2023Long-Range Transformer Architectures for Document Understandingcited by 2

50 most cited of 63 All 63 (JSON)