mlsanityData to 7 Oct 2026

DocTr: Document Transformer for Structured Information Extraction in Documents

arXiv 2307.07929 ↗First posted 16 Jul 2023cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

We present a new formulation for structured information extraction (SIE) from visually rich documents. It aims to address the limitations of existing IOB tagging or graph-based formulations, which are either overly reliant on the correct ordering of input text or struggle with decoding a complex graph. Instead, motivated by anchor-based object detectors in vision, we represent an entity as an anchor word and a bounding box, and represent entity linking as the association between anchor words. This is more robust to text ordering, and maintains a compact graph for entity linking. The formulation motivates us to introduce 1) a DOCument TRansformer (DocTr) that aims at detecting and associating entity bounding boxes in visually rich documents, and 2) a simple pre-training strategy that helps learn entity detection in the context of language. Evaluations on three SIE benchmarks show the effectiveness of the proposed formulation, and the overall approach outperforms existing solutions.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 16 Jul 2023.

Builds on

37 of 52 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2019Towards VQA Models That Can Readcited by 1,696
  7. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  8. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  9. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  10. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  11. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  12. 2021OCR-free Document Understanding Transformercited by 400
25 more
  1. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  2. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  3. 2018Graph R-CNN for Scene Graph Generationcited by 296
  4. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  5. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 146
  6. 2019Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQAcited by 107
  7. 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
  8. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  9. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  10. 2020CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documentscited by 98
  11. 2022TableFormer: Table Structure Understanding with Transformerscited by 93
  12. 2017Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Networkcited by 92
  13. 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
  14. 2021StrucTexT: Structured Text Understanding with Multi-Modal Transformerscited by 90
  15. 2019Graph Convolution for Multimodal Information Extraction from Visually Rich Documentscited by 88
  16. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  17. 2020PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networkscited by 70
  18. 2020Spatial Dependency Parsing for Semi-Structured Document Information Extractioncited by 65
  19. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  20. 2019BERTgrid: Contextualized Embedding for 2D Document Representation and Understandingcited by 63
  21. 2020TRIE: End-to-End Text Reading and Information Extraction for Document Understandingcited by 61
  22. 2021Document AI: Benchmarks, Models and Applicationscited by 61
  23. 2020LAMBERT: Layout-Aware (Language) Modeling for information extractioncited by 53
  24. 2020Structured Multimodal Attentions for TextVQAcited by 23
  25. 2020Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Modelscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 5; 2026: 3.

18 papers.

Sort:

By area:

  1. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 108
  2. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  3. 2023PatFig: Generating Short and Long Captions for Patent Figurescited by 12
  4. 2024Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understandingcited by 12
  5. 2024Hierarchical Visual Feature Aggregation for OCR-Free Document Understandingcited by 12
  6. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMscited by 7
  7. 2024Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Reviewcited by 6
  8. 2024Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extractioncited by 3
  9. 2024CREPE: Coordinate-Aware End-to-End Document Parsercited by 3
  10. 2024PEneo: Unifying Line Extraction, Line Grouping, and Entity Linking for End-to-end Document Pair Extractioncited by 2
  11. 2024GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creationcited by 2
  12. 2024DistilDoc: Knowledge Distillation for Visually-Rich Document Applicationscited by 1
6 more
  1. 2025E-ARMOR: Edge case Assessment and Review of Multilingual Optical Character Recognitioncited by 1
  2. 2025Seeing Straight: Document Orientation Detection for Efficient OCRcited by 1
  3. 2025RoBiologyDataChoiceQA: A Romanian Dataset for improving Biology understanding of Large Language Modelscited by 0
  4. 2026Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Applicationcited by 0
  5. 2026Impact of Iterative Fine-Tuning on Transcription Accuracy in Complex Historical Sanskrit Manuscriptscited by 0
  6. 2026MoTE: Mixture of Task Experts for Multi-Task Video Understandingcited by 0