mlsanityData to 14 Oct 2026

ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-training

arXiv 2410.10471 ↗First posted 14 Oct 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent approaches for visually-rich document understanding (VrDU) uses manually annotated semantic groups, where a semantic group encompasses all semantically relevant but not obviously grouped words. As OCR tools are unable to automatically identify such grouping, we argue that current VrDU approaches are unrealistic. We thus introduce a new variant of the VrDU task, real-world visually-rich document understanding (ReVrDU), that does not allow for using manually annotated semantic groups. We also propose a new method, ReLayout, compliant with the ReVrDU scenario, which learns to capture semantic grouping through arranging words and bringing the representations of words that belong to the potential same semantic group closer together. Our experimental results demonstrate the performance of existing methods is deteriorated with the ReVrDU task, while ReLayout shows superiour performance.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 29 of its 37 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

29 of 37 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  7. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  8. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  9. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  10. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  11. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  12. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
17 more
  1. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  2. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  3. 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
  4. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  5. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  6. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  7. 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
  8. 2021StrucTexT: Structured Text Understanding with Multi-Modal Transformerscited by 90
  9. 2019Graph Convolution for Multimodal Information Extraction from Visually Rich Documentscited by 88
  10. 2022Unified Pretraining Framework for Document Understandingcited by 85
  11. 2021LayoutReader: Pre-training of Text and Layout for Reading Order Detectioncited by 80
  12. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  13. 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
  14. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  15. 2019BERTgrid: Contextualized Embedding for 2D Document Representation and Understandingcited by 63
  16. 2020LAMBERT: Layout-Aware (Language) Modeling for information extractioncited by 53
  17. 2023LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understandingcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMscited by 6
  2. 2025DocReward: A Document Reward Model for Structuring and Stylizingcited by 1
  3. 2026Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesiscited by 1
  4. 2025DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structurescited by 0