# Document AI: Benchmarks, Models and Applications

arXiv: https://arxiv.org/abs/2111.08609 · HTML: https://mlsanity.com/paper/2111.08609/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 117 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval](https://mlsanity.com/paper/1502.07058/) (2015)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [VisualMRC: Machine Reading Comprehension on Document Images](https://mlsanity.com/paper/2101.11272/) (2021)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [SelfDoc: Self-Supervised Document Representation Learning](https://mlsanity.com/paper/2106.03331/) (2021)
- [LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding](https://mlsanity.com/paper/2104.08836/) (2021)
- [TableBank: A Benchmark Dataset for Table Detection and Recognition](https://mlsanity.com/paper/1903.01949/) (2019)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [WebSRC: A Dataset for Web-Based Structural Reading Comprehension](https://mlsanity.com/paper/2101.09465/) (2021)
- [CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents](https://mlsanity.com/paper/2004.12629/) (2020)

## Built on by (most cited first, 30 of 62)

- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [DiT: Self-supervised Pre-training for Document Image Transformer](https://mlsanity.com/paper/2203.02378/) (2022)
- [DocLLM: A layout-aware generative language model for multimodal document understanding](https://mlsanity.com/paper/2401.00908/) (2023)
- [KOSMOS-2.5: A Multimodal Literate Model](https://mlsanity.com/paper/2309.11419/) (2023)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [Towards Complex Document Understanding By Discrete Reasoning](https://mlsanity.com/paper/2207.11871/) (2022)
- [OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation](https://mlsanity.com/paper/2407.19056/) (2024)
- [Vision Grid Transformer for Document Layout Analysis](https://mlsanity.com/paper/2308.14978/) (2023)
- [MMDocIR: Benchmarking Multimodal Retrieval for Long Documents](https://mlsanity.com/paper/2501.08828/) (2025)
- [GeoLayoutLM: Geometric Pre-training for Visual Information Extraction](https://mlsanity.com/paper/2304.10759/) (2023)
- [HRVDA: High-Resolution Visual Document Assistant](https://mlsanity.com/paper/2404.06918/) (2024)
- [DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming](https://mlsanity.com/paper/2406.19101/) (2024)
- [A Survey of Table Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08259/) (2024)
- [HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures](https://mlsanity.com/paper/2303.13839/) (2023)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval](https://mlsanity.com/paper/2506.12364/) (2025)
- [Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding](https://mlsanity.com/paper/2212.09621/) (2022)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [DocTr: Document Transformer for Structured Information Extraction in Documents](https://mlsanity.com/paper/2307.07929/) (2023)
- [Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentration](https://mlsanity.com/paper/2309.01131/) (2023)
- [GridFormer: Towards Accurate Table Structure Recognition via Grid Prediction](https://mlsanity.com/paper/2309.14962/) (2023)
- [XDoc: Unified Pre-training for Cross-Format Document Understanding](https://mlsanity.com/paper/2210.02849/) (2022)
- [ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding](https://mlsanity.com/paper/2209.08569/) (2022)
- [A Simple yet Effective Layout Token in Large Language Models for Document Understanding](https://mlsanity.com/paper/2503.18434/) (2025)
- [A Graphical Approach to Document Layout Analysis](https://mlsanity.com/paper/2308.02051/) (2023)
- [DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyond](https://mlsanity.com/paper/2310.12430/) (2023)
- [Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval](https://mlsanity.com/paper/2602.19961/) (2026)
- [Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges](https://mlsanity.com/paper/2309.14084/) (2023)
- [A Hybrid Approach for Document Layout Analysis in Document images](https://mlsanity.com/paper/2404.17888/) (2024)
- [Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Review](https://mlsanity.com/paper/2408.06345/) (2024)

## Abstract (arXiv)

Document AI, or Document Intelligence, is a relatively new research topic that refers to the techniques for automatically reading, understanding, and analyzing business documents. It is an important research direction for natural language processing and computer vision. In recent years, the popularity of deep learning technology has greatly advanced the development of Document AI, such as document layout analysis, visual information extraction, document visual question answering, document image classification, etc. This paper briefly reviews some of the representative models, tasks, and benchmark datasets. Furthermore, we also introduce early-stage heuristic rule-based document analysis, statistical machine learning algorithms, and deep learning approaches especially pre-training methods. Finally, we look into future directions for Document AI research.