# LAMBERT: Layout-Aware (Language) Modeling for information extraction

arXiv: https://arxiv.org/abs/2002.08087 · HTML: https://mlsanity.com/paper/2002.08087/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 30 of 53)

- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [DiT: Self-supervised Pre-training for Document Image Transformer](https://mlsanity.com/paper/2203.02378/) (2022)
- [Unifying Vision, Text, and Layout for Universal Document Processing](https://mlsanity.com/paper/2212.02623/) (2022)
- [Hierarchical multimodal transformers for Multi-Page DocVQA](https://mlsanity.com/paper/2212.05935/) (2022)
- [OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation](https://mlsanity.com/paper/2407.19056/) (2024)
- [XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding](https://mlsanity.com/paper/2203.06947/) (2022)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding](https://mlsanity.com/paper/2110.08518/) (2021)
- [Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks](https://mlsanity.com/paper/2201.09745/) (2022)
- [LMDX: Language Model-based Document Information Extraction and Localization](https://mlsanity.com/paper/2309.10952/) (2023)
- [VRDU: A Benchmark for Visually-rich Document Understanding](https://mlsanity.com/paper/2211.15421/) (2022)
- [ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents](https://mlsanity.com/paper/2105.11672/) (2021)
- [HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Information](https://mlsanity.com/paper/2203.09629/) (2022)
- [Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering](https://mlsanity.com/paper/2306.00526/) (2023)
- [LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understanding](https://mlsanity.com/paper/2403.14252/) (2024)
- [Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction](https://mlsanity.com/paper/2310.11016/) (2023)
- [DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding](https://mlsanity.com/paper/2408.15045/) (2024)
- [CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data](https://mlsanity.com/paper/2304.14953/) (2023)
- [FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction](https://mlsanity.com/paper/2305.02549/) (2023)
- [DocParser: End-to-end OCR-free Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2304.12484/) (2023)
- [DocTr: Document Transformer for Structured Information Extraction in Documents](https://mlsanity.com/paper/2307.07929/) (2023)
- [STable: Table Generation Framework for Encoder-Decoder Models](https://mlsanity.com/paper/2206.04045/) (2022)
- [XDoc: Unified Pre-training for Cross-Format Document Understanding](https://mlsanity.com/paper/2210.02849/) (2022)
- [Business Document Information Extraction: Towards Practical Benchmarks](https://mlsanity.com/paper/2206.11229/) (2022)
- [Towards Few-shot Entity Recognition in Document Images: A Label-aware Sequence-to-Sequence Framework](https://mlsanity.com/paper/2204.05819/) (2022)
- [ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding](https://mlsanity.com/paper/2209.08569/) (2022)
- [MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding](https://mlsanity.com/paper/2211.14958/) (2022)
- [A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images](https://mlsanity.com/paper/2304.07957/) (2023)

## Abstract (arXiv)

We introduce a simple new approach to the problem of understanding documents where non-trivial layout influences the local semantics. To this end, we modify the Transformer encoder architecture in a way that allows it to use layout features obtained from an OCR system, without the need to re-learn language semantics from scratch. We only augment the input of the model with the coordinates of token bounding boxes, avoiding, in this way, the use of raw images. This leads to a layout-aware language model which can then be fine-tuned on downstream tasks. The model is evaluated on an end-to-end information extraction task using four publicly available datasets: Kleister NDA, Kleister Charity, SROIE and CORD. We show that our model achieves superior performance on datasets consisting of visually rich documents, while also outperforming the baseline RoBERTa on documents with flat layout (NDA (F1) increase from 78.50 to 80.42). Our solution ranked first on the public leaderboard for the Key Information Extraction from the SROIE dataset, improving the SOTA (F1)-score from 97.81 to 98.17.