mlsanityData to 7 Oct 2026

DocILE Benchmark for Document Information Localization and Extraction

arXiv 2302.05658 ↗First posted 11 Feb 2023cs.CL cs.AI cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

This paper introduces the DocILE benchmark with the largest dataset of business documents for the tasks of Key Information Localization and Extraction and Line Item Recognition. It contains 6.7k annotated business documents, 100k synthetically generated documents, and nearly~1M unlabeled documents for unsupervised pre-training. The dataset has been built with knowledge of domain- and task-specific aspects, resulting in the following key features: (i) annotations in 55 classes, which surpasses the granularity of previously published key information extraction datasets by a large margin; (ii) Line Item Recognition represents a highly practical information extraction task, where key information has to be assigned to items in a table; (iii) documents come from numerous layouts and the test set includes zero- and few-shot cases as well as layouts commonly seen in the training set. The benchmark comes with several baselines, including RoBERTa, LayoutLMv3 and DETR-based Table Transformer; applied to both tasks of the DocILE benchmark, with results shared in this paper, offering a quick starting point for future work. The dataset, baselines and supplementary material are available at https://github.com/rossumai/docile.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 67 of its 107 references that are in the map. First posted on arXiv 11 Feb 2023.

Builds on

67 of 107 references.

References · 67

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  9. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  11. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  12. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
38 more
  1. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  2. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  3. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  4. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  5. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  6. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  7. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 281
  8. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  9. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  10. 2019Real-time Scene Text Detection with Differentiable Binarizationcited by 212
  11. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  12. 2020PP-OCR: A Practical Ultra Lightweight OCR Systemcited by 147
  13. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 146
  14. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  15. 2019ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019cited by 138
  16. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  17. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 122
  18. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  19. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  20. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  21. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  22. 2022TableFormer: Table Structure Understanding with Transformerscited by 93
  23. 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
  24. 2021StrucTexT: Structured Text Understanding with Multi-Modal Transformerscited by 90
  25. 2022Unified Pretraining Framework for Document Understandingcited by 85
  26. 2021Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layoutscited by 82
  27. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  28. 2020Spatial Dependency Parsing for Semi-Structured Document Information Extractioncited by 65
  29. 2021Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solutioncited by 64
  30. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  31. 2019BERTgrid: Contextualized Embedding for 2D Document Representation and Understandingcited by 63
  32. 2020Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)cited by 56
  33. 2020LAMBERT: Layout-Aware (Language) Modeling for information extractioncited by 53
  34. 2021Spatial Dual-Modality Graph Reasoning for Key Information Extractioncited by 43
  35. 2018Extracting Scientific Figures with Distantly Supervised Neural Networkscited by 36
  36. 2022OCR-IDL: OCR Annotations for Industry Document Library Datasetcited by 35
  37. 2021ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documentscited by 29
  38. 2018Attend, Copy, Parse -- End-to-end information extraction from documentscited by 25

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 19; 2025: 14; 2026: 20.

58 papers.

Sort:

By area:

  1. 2024A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understandingcited by 57
  2. 2024DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimmingcited by 35
  3. 2024InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructionscited by 27
  4. 2024DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understandingcited by 23
  5. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  6. 2024MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understandingcited by 18
  7. 2023Privacy-Aware Document Visual Question Answeringcited by 17
  8. 2023Towards reducing hallucination in extracting information from financial reports using Large Language Modelscited by 11
  9. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  10. 2024ANLS* -- A Universal Document Processing Metric for Generative Large Language Modelscited by 10
  11. 2024BuDDIE: A Business Document Dataset for Multi-task Information Extractioncited by 8
  12. 2025A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trendscited by 8
38 more
  1. 2024Multimodal Structured Generation: CVPR's 2nd MMFM Challenge Technical Reportcited by 7
  2. 2025Visual Template Inference for Data Extraction from Documentscited by 6
  3. 2024ProcTag: Process Tagging for Assessing the Efficacy of Document Instruction Datacited by 5
  4. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  5. 2026DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splittingcited by 5
  6. 2024"What is the value of {templates}?" Rethinking Document Information Extraction Datasets for LLMscited by 4
  7. 2025Multi-Agent Interactive Question Generation Framework for Long Document Understandingcited by 4
  8. 2026Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligencecited by 4
  9. 2026UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documentscited by 4
  10. 2026VAREX: A Benchmark for Multi-Modal Structured Extraction from Documentscited by 4
  11. 2024Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extractioncited by 3
  12. 2024TreeForm: End-to-end Annotation and Evaluation for Form Document Parsingcited by 3
  13. 2024Retrieval Augmented Structured Generation: Business Document Information Extraction As Tool Usecited by 3
  14. 2025RAPTOR: Refined Approach for Product Table Object Recognitioncited by 3
  15. 2026Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automationcited by 3
  16. 2023PrIeD-KIE: Towards Privacy Preserved Document Key Information Extractioncited by 2
  17. 2024NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQAcited by 2
  18. 2025Relation-Rich Visual Document Generator for Visual Information Extractioncited by 2
  19. 2025QUEST: Quality-aware Semi-supervised Table Extraction for Business Documentscited by 2
  20. 2025FlexDoc: Parameterized Sampling for Diverse Multilingual Synthetic Documents for Training Document Understanding Modelscited by 2
  21. 2026Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extractioncited by 2
  22. 2026GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documentscited by 2
  23. 2024DistilDoc: Knowledge Distillation for Visually-Rich Document Applicationscited by 1
  24. 2024UNER: A Unified Prediction Head for Named Entity Recognition in Visually-rich Documentscited by 1
  25. 2024Training LayoutLM from Scratch for Efficient Named-Entity Recognition in the Insurance Domaincited by 1
  26. 2025Enhancing Document Key Information Localization Through Data Augmentationcited by 1
  27. 2026ICDAR 2025 Competition on End-to-End Document Image Machine Translation Towards Complex Layoutscited by 1
  28. 2026Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesiscited by 1
  29. 2026Pre-Inference Routing for Cost-Efficient Document Field Extractioncited by 1
  30. 2023Lazy-k: Decoding for Constrained Token Classificationcited by 0
  31. 2024UniVIE: A Unified Label Space Approach to Visual Information Extraction from Form-like Documentscited by 0
  32. 2025Generative Compositor for Few-Shot Visual Information Extractioncited by 0
  33. 2025DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structurescited by 0
  34. 2025TASER: Table Agents for Schema-guided Extraction and Recommendationcited by 0
  35. 2025Invoice Information Extraction: Methods and Performance Evaluationcited by 0
  36. 2025Neurosymbolic Information Extraction from Transactional Documentscited by 0
  37. 2026Key Coverage Matters: Semi-Structured Extraction of OCR Clinical Reportscited by 0
  38. 2026BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehensioncited by 0

50 most cited of 58 All 58 (JSON)