# Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout

arXiv: https://arxiv.org/abs/2003.02356 · HTML: https://mlsanity.com/paper/2003.02356/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 27 of 27)

- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding](https://mlsanity.com/paper/2104.08836/) (2021)
- [Towards Complex Document Understanding By Discrete Reasoning](https://mlsanity.com/paper/2207.11871/) (2022)
- [ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding](https://mlsanity.com/paper/2210.06155/) (2022)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding](https://mlsanity.com/paper/2110.08518/) (2021)
- [Entity Relation Extraction as Dependency Parsing in Visually Rich Documents](https://mlsanity.com/paper/2110.09915/) (2021)
- [Form2Seq : A Framework for Higher-Order Form Structure Extraction](https://mlsanity.com/paper/2107.04419/) (2021)
- [FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding](https://mlsanity.com/paper/2505.17330/) (2025)
- [Understanding Long Documents with Different Position-Aware Attentions](https://mlsanity.com/paper/2208.08201/) (2022)
- [Doc2Dict: Information Extraction as Text Generation](https://mlsanity.com/paper/2105.07510/) (2021)
- [TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2207.06744/) (2022)
- [Detection Masking for Improved OCR on Noisy Documents](https://mlsanity.com/paper/2205.08257/) (2022)
- [DocReader: Bounding-Box Free Training of a Document Information Extraction Model](https://mlsanity.com/paper/2105.04313/) (2021)
- [LoRaLay: A Multilingual and Multimodal Dataset for Long Range and Layout-Aware Summarization](https://mlsanity.com/paper/2301.11312/) (2023)
- [Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents](https://mlsanity.com/paper/2306.01058/) (2023)
- [Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding](https://mlsanity.com/paper/2402.18262/) (2024)
- [DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction](https://mlsanity.com/paper/2103.05908/) (2021)
- [Classification of Visualization Types and Perspectives in Patents](https://mlsanity.com/paper/2307.10471/) (2023)
- [Information Extraction from Visually Rich Documents with Font Style Embeddings](https://mlsanity.com/paper/2111.04045/) (2021)
- [Unimodal and Multimodal Representation Training for Relation Extraction](https://mlsanity.com/paper/2211.06168/) (2022)
- [RDU: A Region-based Approach to Form-style Document Understanding](https://mlsanity.com/paper/2206.06890/) (2022)
- [Estimating Post-OCR Denoising Complexity on Numerical Texts](https://mlsanity.com/paper/2307.01020/) (2023)
- [The future of document indexing: GPT and Donut revolutionize table of content processing](https://mlsanity.com/paper/2403.07553/) (2024)
- [Judge a Book by its Cover: Investigating Multi-Modal LLMs for Multi-Page Handwritten Document Transcription](https://mlsanity.com/paper/2502.20295/) (2025)
- [RE2: Region-Aware Relation Extraction from Visually Rich Documents](https://mlsanity.com/paper/2305.14590/) (2023)

## Abstract (arXiv)

State-of-the-art solutions for Natural Language Processing (NLP) are able to capture a broad range of contexts, like the sentence-level context or document-level context for short documents. But these solutions are still struggling when it comes to longer, real-world documents with the information encoded in the spatial structure of the document, such as page elements like tables, forms, headers, openings or footers; complex page layout or presence of multiple pages. To encourage progress on deeper and more complex Information Extraction (IE) we introduce a new task (named Kleister) with two new datasets. Utilizing both textual and structural layout features, an NLP system must find the most important information, about various types of entities, in long formal documents. We propose Pipeline method as a text-only baseline with different Named Entity Recognition architectures (Flair, BERT, RoBERTa). Moreover, we checked the most popular PDF processing tools for text extraction (pdf2djvu, Tesseract and Textract) in order to analyze behavior of IE system in presence of errors introduced by these tools.