mlsanityData to 7 Oct 2026

Unified Pretraining Framework for Document Understanding

arXiv 2204.10939 ↗First posted 22 Apr 2022cs.CL cs.CVcited by 85

Not on the map. Why?

Abstract (arXiv)

Document intelligence automates the extraction of information from documents and supports many business applications. Recent self-supervised learning methods on large-scale unlabeled document datasets have opened up promising directions towards reducing annotation efforts by training models with self-supervised objectives. However, most of the existing document pretraining methods are still language-dominated. We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. Each input element is composed of words and visual features from a semantic region of the input document image. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses, encouraging the representation to model sentences, learn similarities, and align modalities. Extensive empirical analysis demonstrates that the pretraining procedure learns better joint representations and leads to improvements in downstream tasks.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 22 Apr 2022.

Builds on

25 of 34 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017Mask R-CNNcited by 7,259
  4. 2016Layer Normalizationcited by 5,579
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  8. 2015Fast R-CNNcited by 2,930
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  12. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
13 more
  1. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  2. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  3. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  4. 2019Similarity-Preserving Knowledge Distillationcited by 507
  5. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  6. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  7. 2015Recent Advances in Convolutional Neural Networkscited by 360
  8. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  9. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  10. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  11. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  12. 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
  13. 2019READ: Recursive Autoencoders for Document Layout Generationcited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 33; 2024: 27; 2025: 10; 2026: 5.

85 papers.

Sort:

By area:

  1. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 144
  2. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  3. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  4. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 94
  5. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  6. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  7. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  8. 2023Vision Grid Transformer for Document Layout Analysiscited by 45
  9. 2023GeoLayoutLM: Geometric Pre-training for Visual Information Extractioncited by 40
  10. 2023StrucTexTv2: Masked Visual-Textual Prediction for Document Image Pre-trainingcited by 39
  11. 2025SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinementcited by 27
  12. 2024LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understandingcited by 24
38 more
  1. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 22
  2. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  3. 2022Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understandingcited by 20
  4. 2023Modeling Entities as Semantic Points for Visual Information Extraction in the Wildcited by 20
  5. 2023FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extractioncited by 20
  6. 2022Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysiscited by 17
  7. 2022Evaluating Out-of-Distribution Performance on Document Image Classifierscited by 16
  8. 2024Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysiscited by 14
  9. 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posterscited by 14
  10. 2023SwinDocSegmenter: An End-to-End Unified Domain Adaptive Transformer for Document Instance Segmentationcited by 13
  11. 2022XDoc: Unified Pre-training for Cross-Format Document Understandingcited by 12
  12. 2024Visually Guided Generative Text-Layout Pre-training for Document Intelligencecited by 12
  13. 2025Handwritten Text Recognition: A Surveycited by 12
  14. 2022Relational Representation Learning in Visually-Rich Documentscited by 11
  15. 2024DLAFormer: An End-to-End Transformer For Document Layout Analysiscited by 11
  16. 2023Beyond Document Page Classification: Design, Datasets, and Challengescited by 10
  17. 2022MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understandingcited by 9
  18. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  19. 2023A Graphical Approach to Document Layout Analysiscited by 8
  20. 2024Multi-Page Document Visual Question Answering using Self-Attention Scoring Mechanismcited by 8
  21. 2023SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentationcited by 7
  22. 2023DocumentCLIP: Linking Figures and Main Body Text in Reflowed Documentscited by 7
  23. 2023Bridging the Performance Gap between DETR and R-CNN for Graphical Object Detection in Document Imagescited by 7
  24. 2023RealCQA: Scientific Chart Question Answering as a Test-bed for First-Order Logiccited by 7
  25. 2024PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answeringcited by 7
  26. 2022Test-Time Adaptation for Visual Document Understandingcited by 6
  27. 2023Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understandingcited by 6
  28. 2023Do-GOOD: Towards Distribution Shift Evaluation for Pre-Trained Visual Document Understanding Modelscited by 6
  29. 2023DSG: An End-to-End Document Structure Generatorcited by 6
  30. 2024Transformers and Language Models in Form Understanding: A Comprehensive Review of Scanned Document Analysiscited by 6
  31. 2024A Hybrid Approach for Document Layout Analysis in Document imagescited by 6
  32. 2024Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Reviewcited by 6
  33. 2023Text Reading Order in Uncontrolled Conditions by Sparse Graph Segmentationcited by 5
  34. 2023On Evaluation of Document Classification using RVL-CDIPcited by 5
  35. 2024ProcTag: Process Tagging for Assessing the Efficacy of Document Instruction Datacited by 5
  36. 2023DocumentNet: Bridging the Data Gap in Document Pre-Trainingcited by 4
  37. 2023FATURA: A Multi-Layout Invoice Image Dataset for Document Analysis and Understandingcited by 4
  38. 2024SynJAC: Synthetic-data-driven Joint-granular Adaptation and Calibration for Domain Specific Scanned Document Key Information Extractioncited by 4

50 most cited of 85 All 85 (JSON)