mlsanityData to 7 Oct 2026

ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction

arXiv 2106.10786 ↗First posted 21 Jun 2021cs.CL cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Natural reading orders of words are crucial for information extraction from form-like documents. Despite recent advances in Graph Convolutional Networks (GCNs) on modeling spatial layout patterns of documents, they have limited ability to capture reading orders of given word-level node representations in a graph. We propose Reading Order Equivariant Positional Encoding (ROPE), a new positional encoding technique designed to apprehend the sequential presentation of words in documents. ROPE generates unique reading order codes for neighboring words relative to the target word given a word-level graph connectivity. We study two fundamental document entity extraction tasks including word labeling and word grouping on the public FUNSD dataset and a large-scale payment dataset. We show that ROPE consistently improves existing GCNs with a margin up to 8.4% F1-score.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 21 Jun 2021.

Builds on

24 of 29 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2017Graph Attention Networkscited by 8,854
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2019Searching for MobileNetV3cited by 1,984
  6. 2018Self-Attention with Relative Position Representationscited by 1,112
  7. 2018Image Transformercited by 770
  8. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  9. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  10. 2017Convolutional Sequence to Sequence Learningcited by 550
  11. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
  12. 2018Graph Convolutional Networks for Text Classificationcited by 345
12 more
  1. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  2. 2019Position-aware Graph Neural Networkscited by 203
  3. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  4. 2019Graph Convolution for Multimodal Information Extraction from Visually Rich Documentscited by 88
  5. 2020PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networkscited by 70
  6. 2020Learning to Encode Position for Transformer with Continuous Dynamical Modelcited by 61
  7. 2017Graph-based Neural Multi-Document Summarizationcited by 54
  8. 2018GraphIE: A Graph-Based Framework for Information Extractioncited by 45
  9. 2017CloudScan - A configuration-free invoice analysis system using recurrent neural networkscited by 33
  10. 2019Deep Visual Template-Free Form Parsingcited by 24
  11. 2019CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractorcited by 21
  12. 2021Post-OCR Paragraph Recognition by Graph Convolutional Networkscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 8; 2024: 5; 2025: 2; 2026: 1.

22 papers.

Sort:

By area:

  1. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  2. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 56
  3. 2023LMDX: Language Model-based Document Information Extraction and Localizationcited by 37
  4. 2022VRDU: A Benchmark for Visually-rich Document Understandingcited by 35
  5. 2023FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extractioncited by 20
  6. 2024CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialoguescited by 15
  7. 2023End-to-End Page-Level Assessment of Handwritten Text Recognitioncited by 8
  8. 2022Unified Line and Paragraph Detection by Graph Convolutional Networkscited by 7
  9. 2022QueryForm: A Simple Zero-shot Form Entity Query Frameworkcited by 7
  10. 2023Enhancing Visually-Rich Document Understanding via Layout Structure Modelingcited by 6
  11. 2024DocGraphLM: Documental Graph Language Model for Information Extractioncited by 6
  12. 2023Text Reading Order in Uncontrolled Conditions by Sparse Graph Segmentationcited by 5
10 more
  1. 2024Hypergraph based Understanding for Document Semantic Entity Recognitioncited by 4
  2. 2025Where is this coming from? Making groundedness count in the evaluation of Document VQA modelscited by 4
  3. 2025Pre-Training LLMs on a budget: A comparison of three optimizerscited by 4
  4. 2024Unsupervised Extraction of Dialogue Policies from Conversationscited by 3
  5. 2023PARAGRAPH2GRAPH: A GNN-based framework for layout paragraph analysiscited by 2
  6. 2023DocTrack: A Visually-Rich Document Dataset Really Aligned with Human Eye Movement for Machine Readingcited by 2
  7. 2026RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wildcited by 2
  8. 2022Towards Optimizing OCR for Accessibilitycited by 1
  9. 2023GVdoc: Graph-based Visual Document Classificationcited by 1
  10. 2024HIP: Hierarchical Point Modeling and Pre-training for Visual Information Extractioncited by 0