mlsanityData to 14 Oct 2026

LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding

arXiv 2104.08836 ↗First posted 18 Apr 2021cs.CLcited by 103

Not on the map. Why?

Abstract (arXiv)

Multimodal pre-training with text, layout, and image has achieved SOTA performance for visually-rich document understanding tasks recently, which demonstrates the great potential for joint learning across different modalities. In this paper, we present LayoutXLM, a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding. To accurately evaluate LayoutXLM, we also introduce a multilingual form understanding benchmark dataset named XFUND, which includes form understanding samples in 7 languages (Chinese, Japanese, Spanish, French, Italian, German, Portuguese), and key-value pairs are manually labeled for each language. Experiment results show that the LayoutXLM model has significantly outperformed the existing SOTA cross-lingual pre-trained models on the XFUND dataset. The pre-trained LayoutXLM model and the XFUND dataset are publicly available at https://aka.ms/layoutxlm.

Key facts. Cited by 103 papers in the map (data through 14 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

23 of 28 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  6. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  8. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  9. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  12. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
11 more
  1. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  2. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  3. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  4. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  5. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  6. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  7. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 213
  8. 2020DocBank: A Benchmark Dataset for Document Layout Analysiscited by 153
  9. 2019TableBank: A Benchmark Dataset for Table Detection and Recognitioncited by 102
  10. 2018Joint entity recognition and relation extraction as a multi-head selection problemcited by 59
  11. 2020Kleister: A novel task for Information Extraction involving Long Documents with Complex Layoutcited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 22; 2023: 27; 2024: 20; 2025: 16; 2026: 14.

103 papers.

Sort:

By area:

  1. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  2. 2021OCR-free Document Understanding Transformercited by 407
  3. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  4. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  5. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  6. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  7. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  8. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  9. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  10. 2021Document AI: Benchmarks, Models and Applicationscited by 62
  11. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  12. 2021MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understandingcited by 45
38 more
  1. 2023GeoLayoutLM: Geometric Pre-training for Visual Information Extractioncited by 40
  2. 2023StrucTexTv2: Masked Visual-Textual Prediction for Document Image Pre-trainingcited by 39
  3. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 26
  4. 2023Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answeringcited by 24
  5. 2023Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Predictioncited by 23
  6. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 23
  7. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  8. 2023CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Datacited by 21
  9. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  10. 2022PP-StructureV2: A Stronger Document Analysis Systemcited by 20
  11. 2022Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understandingcited by 20
  12. 2023Modeling Entities as Semantic Points for Visual Information Extraction in the Wildcited by 20
  13. 2023A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analyticscited by 18
  14. 2023Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentrationcited by 18
  15. 2022XDoc: Unified Pre-training for Cross-Format Document Understandingcited by 12
  16. 2024Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understandingcited by 12
  17. 2022Business Document Information Extraction: Towards Practical Benchmarkscited by 11
  18. 2022Towards Few-shot Entity Recognition in Document Images: A Label-aware Sequence-to-Sequence Frameworkcited by 10
  19. 2022ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Selectcited by 10
  20. 2025FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understandingcited by 10
  21. 2022ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understandingcited by 9
  22. 2022Understanding Long Documents with Different Position-Aware Attentionscited by 7
  23. 2022QueryForm: A Simple Zero-shot Form Entity Query Frameworkcited by 7
  24. 2021A Span Extraction Approach for Information Extraction on Visually-Rich Documentscited by 6
  25. 2023Do-GOOD: Towards Distribution Shift Evaluation for Pre-Trained Visual Document Understanding Modelscited by 6
  26. 2023Enhancing Visually-Rich Document Understanding via Layout Structure Modelingcited by 6
  27. 2024Transformers and Language Models in Form Understanding: A Comprehensive Review of Scanned Document Analysiscited by 6
  28. 2022On Web-based Visual Corpus Construction for Visual Document Understandingcited by 5
  29. 2024LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretrainingcited by 5
  30. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  31. 2022Manifestations of Xenophobia in AI Systemscited by 4
  32. 2023Information Redundancy and Biases in Public Document Information Extraction Benchmarkscited by 4
  33. 2023PPN: Parallel Pointer-based Network for Key Information Extraction with Complex Layoutscited by 4
  34. 2024Hypergraph based Understanding for Document Semantic Entity Recognitioncited by 4
  35. 2024ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-trainingcited by 4
  36. 2025Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trendscited by 4
  37. 2026Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligencecited by 4
  38. 2022Sequence-aware multimodal page classification of Brazilian legal documentscited by 3

50 most cited of 103 All 103 (JSON)