mlsanityData to 14 Oct 2026

Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layouts

arXiv 2105.05796 ↗First posted 12 May 2021cs.CLcited by 82

Not on the map. Why?

Cited by 28 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The relevance of the Key Information Extraction (KIE) task is increasingly important in natural language processing problems. But there are still only a few well-defined problems that serve as benchmarks for solutions in this area. To bridge this gap, we introduce two new datasets (Kleister NDA and Kleister Charity). They involve a mix of scanned and born-digital long formal English-language documents. In these datasets, an NLP system is expected to find or infer various types of entities by employing both textual and structural layout features. The Kleister Charity dataset consists of 2,788 annual financial reports of charity organizations, with 61,643 unique pages and 21,612 entities to extract. The Kleister NDA dataset has 540 Non-disclosure Agreements, with 3,229 unique pages and 2,160 entities to extract. We provide several state-of-the-art baseline systems from the KIE domain (Flair, BERT, RoBERTa, LayoutLM, LAMBERT), which show that our datasets pose a strong challenge to existing models. The best model achieved an 81.77% and an 83.57% F1-score on respectively the Kleister NDA and the Kleister Charity datasets. We share the datasets to encourage progress on more in-depth and complex information extraction tasks.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

14 of 26 references.

References · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  5. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  6. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  7. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  8. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  9. 2019Graph Convolution for Multimodal Information Extraction from Visually Rich Documentscited by 88
  10. 2017CloudScan - A configuration-free invoice analysis system using recurrent neural networkscited by 33
  11. 2016WikiReading: A Novel Large-scale Language Understanding Task over Wikipediacited by 26
  12. 2018Attend, Copy, Parse -- End-to-end information extraction from documentscited by 25
2 more
  1. 2020From Dataset Recycling to Multi-Property Extraction and Beyondcited by 6
  2. 2020A framework to evaluate the viability of robotic process automation for business process activitiescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 18; 2024: 31; 2025: 16; 2026: 10.

82 papers.

Sort:

By area:

  1. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  2. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  3. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  4. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
  5. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 168
  6. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  7. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  8. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 135
  9. 2024SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Modelscited by 131
  10. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  11. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
  12. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
38 more
  1. 2024LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMscited by 84
  2. 2021Document AI: Benchmarks, Models and Applicationscited by 62
  3. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  4. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  5. 2024HRVDA: High-Resolution Visual Document Assistantcited by 37
  6. 2022VRDU: A Benchmark for Visually-rich Document Understandingcited by 35
  7. 2024DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimmingcited by 35
  8. 2023mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Modelcited by 32
  9. 2024Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Modelscited by 30
  10. 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
  11. 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
  12. 2023Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Predictioncited by 23
  13. 2024DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understandingcited by 23
  14. 2024Token-level Correlation-guided Compression for Efficient Multimodal Document Understandingcited by 22
  15. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  16. 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
  17. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  18. 2025A Token-level Text Image Foundation Model for Document Understandingcited by 14
  19. 2024Hierarchical Visual Feature Aggregation for OCR-Free Document Understandingcited by 12
  20. 2022Business Document Information Extraction: Towards Practical Benchmarkscited by 11
  21. 2024StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyondcited by 11
  22. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  23. 2024DOGR: Towards Versatile Visual Document Grounding and Referringcited by 10
  24. 2025AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understandingcited by 9
  25. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Reviewcited by 9
  26. 2026ExtractBench: A Benchmark and Evaluation Methodology for Complex Structured Extractioncited by 9
  27. 2023GenKIE: Robust Generative Multimodal Document Key Information Extractioncited by 8
  28. 2024BuDDIE: A Business Document Dataset for Multi-task Information Extractioncited by 8
  29. 2025BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotationscited by 8
  30. 2022Understanding Long Documents with Different Position-Aware Attentionscited by 7
  31. 2023SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentationcited by 7
  32. 2025Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMscited by 6
  33. 2022Information Extraction from Scanned Invoice Images using Text Analysis and Layout Featurescited by 5
  34. 2024KVP10k : A Comprehensive Dataset for Key-Value Pair Extraction in Business Documentscited by 5
  35. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  36. 2026DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splittingcited by 5
  37. 2023Information Redundancy and Biases in Public Document Information Extraction Benchmarkscited by 4
  38. 2023DocumentNet: Bridging the Data Gap in Document Pre-Trainingcited by 4

50 most cited of 82 All 82 (JSON)