mlsanityData to 14 Oct 2026

DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures

arXiv 2507.08606 ↗First posted 11 Jul 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce DocPolarBERT, a layout-aware BERT model for document understanding that eliminates the need for absolute 2D positional embeddings. We extend self-attention to take into account text block positions in relative polar coordinate system rather than the Cartesian one. Despite being pre-trained on a dataset more than six times smaller than the widely used IIT-CDIP corpus, DocPolarBERT achieves state-of-the-art results. These results demonstrate that a carefully designed attention mechanism can compensate for reduced pre-training data, offering an efficient and effective alternative for document understanding.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

30 of 35 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2018Self-Attention with Relative Position Representationscited by 1,120
  7. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  8. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  9. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  10. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  11. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 256
  12. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
18 more
  1. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 213
  2. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  3. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  4. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  5. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  6. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 95
  7. 2022Unified Pretraining Framework for Document Understandingcited by 85
  8. 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
  9. 2020Spatial Dependency Parsing for Semi-Structured Document Information Extractioncited by 65
  10. 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
  11. 2024Length Generalization of Causal Transformers without Position Encodingcited by 47
  12. 2024InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructionscited by 27
  13. 2022Structural generalization is hard for sequence-to-sequence modelscited by 24
  14. 2023ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretationcited by 24
  15. 2022Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understandingcited by 20
  16. 2023LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understandingcited by 17
  17. 2024ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-trainingcited by 4
  18. 2024Training LayoutLM from Scratch for Efficient Named-Entity Recognition in the Insurance Domaincited by 2

Built on by

Not cited yet.