mlsanityData to 14 Oct 2026

LayoutXLM vs. GNN: An Empirical Evaluation of Relation Extraction for Documents

arXiv 2206.10304 ↗First posted 9 May 2022cs.CL cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

This paper investigates the Relation Extraction task in documents by benchmarking two different neural network models: a multi-modal language model (LayoutXLM) and a Graph Neural Network: Edge Convolution Network (ECN). For this benchmark, we use the XFUND dataset, released along with LayoutXLM. While both models reach similar results, they both exhibit very different characteristics. This raises the question on how to integrate various modalities in a neural network: by merging all modalities thanks to additional pretraining (LayoutXLM), or in a cascaded way (ECN). We conclude by discussing some methodological issues that must be considered for new datasets and task definition in the domain of Information Extraction with complex documents.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 12 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

11 of 12 references.

References · 11

  1. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  2. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  3. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  4. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  5. 2017Residual Gated Graph ConvNetscited by 346
  6. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  7. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  8. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  9. 2020LAMBERT: Layout-Aware (Language) Modeling for information extractioncited by 53
  10. 2021Entity Relation Extraction as Dependency Parsing in Visually Rich Documentscited by 23
  11. 2019Comparing Machine Learning Approaches for Table Recognition in Historical Register Bookscited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Language Independent Neuro-Symbolic Semantic Parsing for Form Understandingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction
  2. 2025Relation-Rich Visual Document Generator for Visual Information Extraction
  3. 2023Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction
  4. 2023Document-level Relation Extraction with Cross-sentence Reasoning Graph
  5. 2024Graph-DPEP: Decomposed Plug and Ensemble Play for Few-Shot Document Relation Extraction with Graph-of-Thoughts Reasoning
  6. 2026ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images
  7. 2026PARL: Position-Aware Relation Learning Network for Document Layout Analysis
  8. 2023Revisiting Relation Extraction in the era of Large Language Models