# Hypergraph based Understanding for Document Semantic Entity Recognition

arXiv: https://arxiv.org/abs/2407.06904 · HTML: https://mlsanity.com/paper/2407.06904/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding](https://mlsanity.com/paper/2104.08836/) (2021)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding](https://mlsanity.com/paper/2210.06155/) (2022)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction](https://mlsanity.com/paper/2106.10786/) (2021)
- [Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognition](https://mlsanity.com/paper/2208.03054/) (2022)
- [ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding](https://mlsanity.com/paper/2209.08569/) (2022)

## Built on by (most cited first, 4 of 4)

- [DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding](https://mlsanity.com/paper/2408.15045/) (2024)
- [HomeBench: Evaluating LLMs in Smart Homes with Valid and Invalid Instructions Across Single and Multiple Devices](https://mlsanity.com/paper/2505.19628/) (2025)
- [Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding](https://mlsanity.com/paper/2409.19672/) (2024)
- [MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use](https://mlsanity.com/paper/2511.17881/) (2025)

## Abstract (arXiv)

Semantic entity recognition is an important task in the field of visually-rich document understanding. It distinguishes the semantic types of text by analyzing the position relationship between text nodes and the relation between text content. The existing document understanding models mainly focus on entity categories while ignoring the extraction of entity boundaries. We build a novel hypergraph attention document semantic entity recognition framework, HGA, which uses hypergraph attention to focus on entity boundaries and entity categories at the same time. It can conduct a more detailed analysis of the document text representation analyzed by the upstream model and achieves a better performance of semantic information. We apply this method on the basis of GraphLayoutLM to construct a new semantic entity recognition model HGALayoutLM. Our experiment results on FUNSD, CORD, XFUND and SROIE show that our method can effectively improve the performance of semantic entity recognition tasks based on the original model. The results of HGALayoutLM on FUNSD and XFUND reach the new state-of-the-art results.