mlsanityData to 7 Oct 2026

StructuralLM: Structural Pre-training for Form Understanding

arXiv 2105.11210 ↗First posted 24 May 2021cs.CLcited by 91

Not on the map. Why?

Abstract (arXiv)

Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, while neglecting cell-level layout information that is important for form image understanding. In this paper, we propose a new pre-training approach, StructuralLM, to jointly leverage cell and layout information from scanned documents. Specifically, we pre-train StructuralLM with two new designs to make the most of the interactions of cell and layout information: 1) each cell as a semantic unit; 2) classification of cell positions. The pre-trained StructuralLM achieves new state-of-the-art results in different types of downstream tasks, including form understanding (from 78.95 to 85.14), document visual question answering (from 72.59 to 83.94) and document image classification (from 94.43 to 96.08).

Key facts. Cited by 91 papers in the map (data through 7 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 24 May 2021.

Builds on

17 of 25 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  5. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  6. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  7. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  8. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  9. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  10. 2019StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understandingcited by 108
  11. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  12. 2017Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Networkcited by 92
5 more
  1. 2020TRIE: End-to-End Text Reading and Information Extraction for Document Understandingcited by 61
  2. 2018Multi-granularity hierarchical attention fusion networks for reading comprehension and question answeringcited by 28
  3. 2017Cutting the Error by Half: Investigation of Very Deep CNN and Advanced Training Strategies for Document Image Classificationcited by 22
  4. 2018Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networkscited by 20
  5. 2019Modular Multimodal Architecture for Document Classificationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 24; 2023: 33; 2024: 21; 2025: 4; 2026: 4.

91 papers.

Sort:

By area:

  1. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  2. 2021OCR-free Document Understanding Transformercited by 400
  3. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  4. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  5. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
  6. 2022DiT: Self-supervised Pre-training for Document Image Transformercited by 152
  7. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  8. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  9. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  10. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  11. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 95
  12. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 94
38 more
  1. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  2. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  3. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  4. 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
  5. 2021Document AI: Benchmarks, Models and Applicationscited by 61
  6. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  7. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  8. 2021MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understandingcited by 44
  9. 2023GeoLayoutLM: Geometric Pre-training for Visual Information Extractioncited by 40
  10. 2023StrucTexTv2: Masked Visual-Textual Prediction for Document Image Pre-trainingcited by 39
  11. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  12. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  13. 2023Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answeringcited by 24
  14. 2024LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understandingcited by 24
  15. 2022Multimodal Pre-training Based on Graph Attention Network for Document Understandingcited by 23
  16. 2023ICDAR 2023 Competition on Hierarchical Text Detection and Recognitioncited by 23
  17. 2023Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Predictioncited by 23
  18. 2024DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understandingcited by 23
  19. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 22
  20. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  21. 2022Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understandingcited by 20
  22. 2023Modeling Entities as Semantic Points for Visual Information Extraction in the Wildcited by 20
  23. 2023FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extractioncited by 20
  24. 2022XMP-Font: Self-Supervised Cross-Modality Pre-training for Few-Shot Font Generationcited by 18
  25. 2022Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understandingcited by 18
  26. 2023DocTr: Document Transformer for Structured Information Extraction in Documentscited by 18
  27. 2023LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understandingcited by 17
  28. 2022Evaluating Out-of-Distribution Performance on Document Image Classifierscited by 16
  29. 2022XDoc: Unified Pre-training for Cross-Format Document Understandingcited by 12
  30. 2024Visually Guided Generative Text-Layout Pre-training for Document Intelligencecited by 12
  31. 2024Hierarchical Visual Feature Aggregation for OCR-Free Document Understandingcited by 12
  32. 2022Relational Representation Learning in Visually-Rich Documentscited by 11
  33. 2022ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understandingcited by 9
  34. 2025A Simple yet Effective Layout Token in Large Language Models for Document Understandingcited by 9
  35. 2023A Graphical Approach to Document Layout Analysiscited by 8
  36. 2023Hierarchical Text Spotter for Joint Text Spotting and Layout Analysiscited by 8
  37. 2024BuDDIE: A Business Document Dataset for Multi-task Information Extractioncited by 8
  38. 2022Understanding Long Documents with Different Position-Aware Attentionscited by 7

50 most cited of 91 All 91 (JSON)