mlsanityData to 14 Oct 2026

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding

arXiv 2012.14740 ↗First posted 29 Dec 2020cs.CLcited by 408

Not on the map. Why?

Cited by 49 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents. We propose LayoutLMv2 architecture with new pre-training tasks to model the interaction among text, layout, and image in a single multi-modal framework. Specifically, with a two-stream multi-modal Transformer encoder, LayoutLMv2 uses not only the existing masked visual-language modeling task but also the new text-image alignment and text-image matching tasks, which make it better capture the cross-modality interaction in the pre-training stage. Meanwhile, it also integrates a spatial-aware self-attention mechanism into the Transformer architecture so that the model can fully understand the relative positional relationship among different text blocks. Experiment results show that LayoutLMv2 outperforms LayoutLM by a large margin and achieves new state-of-the-art results on a wide variety of downstream visually-rich document understanding tasks, including FUNSD (0.7895 → 0.8420), CORD (0.9493 → 0.9601), SROIE (0.9524 → 0.9781), Kleister-NDA (0.8340 → 0.8520), RVL-CDIP (0.9443 → 0.9564), and DocVQA (0.7295 → 0.8672). We made our model and code publicly available at urlhttps://aka.ms/layoutlmv2.

Key facts. Cited by 408 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 25; 2022: 66; 2023: 99; 2024: 90; 2025: 69; 2026: 59.

408 papers.

Sort:

By area:

  1. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  3. 2021InfographicVQAcited by 437
  4. 2021OCR-free Document Understanding Transformercited by 407
  5. 2022Multimodal Learning with Transformers: A Surveycited by 400
  6. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  7. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  8. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  9. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  10. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  11. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  12. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
38 more
  1. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
  2. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  3. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  4. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 152
  5. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  6. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  7. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  8. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  9. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  10. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 105
  11. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  12. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  13. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  14. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 95
  15. 2021StrucTexT: Structured Text Understanding with Multi-Modal Transformerscited by 90
  16. 2022Unified Pretraining Framework for Document Understandingcited by 85
  17. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  18. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 82
  19. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  20. 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 79
  21. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  22. 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
  23. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  24. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 72
  25. 2024CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automationcited by 70
  26. 2024OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automationcited by 70
  27. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  28. 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
  29. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  30. 2021Document AI: Benchmarks, Models and Applicationscited by 62
  31. 2022DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognitioncited by 62
  32. 2021LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysiscited by 58
  33. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  34. 2024A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understandingcited by 57
  35. 2023Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solutioncited by 54
  36. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 53
  37. 2024Large Language Model for Table Processing: A Surveycited by 52
  38. 2023Vision Grid Transformer for Document Layout Analysiscited by 47

50 most cited of 408 All 408 (JSON)