mlsanityData to 7 Oct 2026

End-to-End Hierarchical Relation Extraction for Generic Form Understanding

arXiv 2106.00980 ↗First posted 2 Jun 2021cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Form understanding is a challenging problem which aims to recognize semantic entities from the input document and their hierarchical relations. Previous approaches face significant difficulty dealing with the complexity of the task, thus treat these objectives separately. To this end, we present a novel deep neural network to jointly perform both entity detection and link prediction in an end-to-end fashion. Our model extends the Multi-stage Attentional U-Net architecture with the Part-Intensity Fields and Part-Association Fields for link prediction, enriching the spatial information flow with the additional supervision from entity linking. We demonstrate the effectiveness of the model on the Form Understanding in Noisy Scanned Documents (FUNSD) dataset, where our method substantially outperforms the original model and state-of-the-art baselines in both Entity Labeling and Entity Linking task.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

24 of 34 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  6. 2018Link Prediction Based on Graph Neural Networkscited by 829
  7. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  8. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  9. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  10. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  11. 2018Image Generation from Scene Graphscited by 324
  12. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
12 more
  1. 2018Graph R-CNN for Scene Graph Generationcited by 296
  2. 2017Towards Accurate Multi-person Pose Estimation in the Wildcited by 131
  3. 2019PifPaf: Composite Fields for Human Pose Estimationcited by 120
  4. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  5. 2019Graph Convolution for Multimodal Information Extraction from Visually Rich Documentscited by 88
  6. 2017Pixels to Graphs by Associative Embeddingcited by 58
  7. 2017CloudScan - A configuration-free invoice analysis system using recurrent neural networkscited by 33
  8. 2018dhSegment: A generic deep-learning approach for document segmentationcited by 32
  9. 2018Attend, Copy, Parse -- End-to-end information extraction from documentscited by 25
  10. 2020Towards a Multi-modal, Multi-task Learning based Pre-training Framework for Document Representation Learningcited by 21
  11. 2017Robust Spatial Filtering with Graph Convolutional Neural Networkscited by 11
  12. 2018CU-Net: Coupled U-Netscited by 11

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 1; 2025: 1.

7 papers.

  1. 2023Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Predictioncited by 23
  2. 2022Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understandingcited by 18
  3. 2023A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Imagescited by 9
  4. 2025Benchmarking Graph Neural Networks for Document Layout Analysis in Public Affairscited by 6
  5. 2022Unimodal and Multimodal Representation Training for Relation Extractioncited by 2
  6. 2021Text Classification Models for Form Entity Linkingcited by 1
  7. 2024VRDSynth: Synthesizing Programs for Multilingual Visually Rich Document Information Extractioncited by 0