mlsanityData to 7 Oct 2026

LayoutLM: Pre-training of Text and Layout for Document Image Understanding

arXiv 1912.13318 ↗First posted 31 Dec 2019cs.CLcited by 549

Not on the map. Why?

Cited by 64 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pre-training techniques have been verified successfully in a variety of NLP tasks in recent years. Despite the widespread use of pre-training models for NLP applications, they almost exclusively focus on text-level manipulation, while neglecting layout and style information that is vital for document image understanding. In this paper, we propose the LayoutLM to jointly model interactions between text and layout information across scanned document images, which is beneficial for a great number of real-world document image understanding tasks such as information extraction from scanned documents. Furthermore, we also leverage image features to incorporate words' visual information into LayoutLM. To the best of our knowledge, this is the first time that text and layout are jointly learned in a single framework for document-level pre-training. It achieves new state-of-the-art results in several downstream tasks, including form understanding (from 70.72 to 79.27), receipt understanding (from 94.02 to 95.24) and document image classification (from 93.07 to 94.42). The code and pre-trained LayoutLM models are publicly available at urlhttps://aka.ms/layoutlm.

Key facts. Cited by 549 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 63; 2022: 70; 2023: 105; 2024: 99; 2025: 106; 2026: 106.

549 papers.

Sort:

By area:

  1. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  3. 2021InfographicVQAcited by 430
  4. 2021OCR-free Document Understanding Transformercited by 400
  5. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  6. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  7. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 240
  8. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  9. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  10. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  11. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  12. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 167
38 more
  1. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
  2. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  3. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  4. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  5. 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 146
  6. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  7. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  8. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 132
  9. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 120
  10. 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
  11. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  12. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  13. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  14. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  15. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  16. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
  17. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 95
  18. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 94
  19. 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
  20. 2021StrucTexT: Structured Text Understanding with Multi-Modal Transformerscited by 90
  21. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  22. 2021Variational Transformer Networks for Layout Generationcited by 88
  23. 2022Unified Pretraining Framework for Document Understandingcited by 85
  24. 2021Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layoutscited by 82
  25. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  26. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  27. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 80
  28. 2021LayoutReader: Pre-training of Text and Layout for Reading Order Detectioncited by 79
  29. 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 78
  30. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  31. 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
  32. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  33. 2024CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automationcited by 70
  34. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
  35. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 68
  36. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  37. 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
  38. 2023SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphscited by 66

50 most cited of 549 All 549 (JSON)