# HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures

arXiv: https://arxiv.org/abs/2303.13839 · HTML: https://mlsanity.com/paper/2303.13839/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding](https://mlsanity.com/paper/2010.11685/) (2020)
- [Multimodal Pre-training Based on Graph Attention Network for Document Understanding](https://mlsanity.com/paper/2203.13530/) (2022)

## Built on by (most cited first, 23 of 23)

- [Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis](https://mlsanity.com/paper/2401.11874/) (2024)
- [READoc: A Unified Benchmark for Realistic Document Structured Extraction](https://mlsanity.com/paper/2409.05137/) (2024)
- [DLAFormer: An End-to-End Transformer For Document Layout Analysis](https://mlsanity.com/paper/2405.11757/) (2024)
- [DSG: An End-to-End Document Structure Generator](https://mlsanity.com/paper/2310.09118/) (2023)
- [PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering](https://mlsanity.com/paper/2404.12720/) (2024)
- [MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents](https://mlsanity.com/paper/2604.12352/) (2026)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026)
- [DocMamba: Efficient Document Pre-training with State Space Model](https://mlsanity.com/paper/2409.11887/) (2024)
- [UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis](https://mlsanity.com/paper/2503.15893/) (2025)
- [HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering](https://mlsanity.com/paper/2605.29606/) (2026)
- [PubTables-v2: A new large-scale dataset for full-page and multi-page table extraction](https://mlsanity.com/paper/2512.10888/) (2025)
- [MinerU-Popo: Universal Post-Processing Model for Structured Document Parsing](https://mlsanity.com/paper/2605.24973/) (2026)
- [Count, Decode and Fetch: A New Approach to Handwritten Chinese Character Error Correction](https://mlsanity.com/paper/2307.16253/) (2023)
- [AnnoPage Dataset: Dataset of Non-Textual Elements in Documents with Fine-Grained Categorization](https://mlsanity.com/paper/2503.22526/) (2025)
- [MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing](https://mlsanity.com/paper/2605.22100/) (2026)
- [TextBite: A Historical Czech Document Dataset for Logical Page Segmentation](https://mlsanity.com/paper/2503.16664/) (2025)
- [PromptDLA: A Domain-aware Prompt Document Layout Analysis Framework with Descriptive Knowledge as a Cue](https://mlsanity.com/paper/2603.09414/) (2026)
- [M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models](https://mlsanity.com/paper/2605.18774/) (2026)
- [DOSA: A Tree-Guided, Self-Regressive Framework for Long Document Structure Analysis](https://mlsanity.com/paper/2607.22679/) (2026)
- [LongDocBench: Benchmarking TOC Hierarchy and Contextual Relationship Recovery in Long Documents](https://mlsanity.com/paper/2608.15064/) (2026)
- [Robust Hierarchical Structures for Agentic Document Analysis](https://mlsanity.com/paper/2609.33322/) (2026)
- [Heading Hierarchy Reconstruction for Complex and Long Documents with Level-Wise Multimodal Memory](https://mlsanity.com/paper/2610.18238/) (2026)

## Abstract (arXiv)

The problem of document structure reconstruction refers to converting digital or scanned documents into corresponding semantic structures. Most existing works mainly focus on splitting the boundary of each element in a single document page, neglecting the reconstruction of semantic structure in multi-page documents. This paper introduces hierarchical reconstruction of document structures as a novel task suitable for NLP and CV fields. To better evaluate the system performance on the new task, we built a large-scale dataset named HRDoc, which consists of 2,500 multi-page documents with nearly 2 million semantic units. Every document in HRDoc has line-level annotations including categories and relations obtained from rule-based extractors and human annotators. Moreover, we proposed an encoder-decoder-based hierarchical document structure parsing system (DSPS) to tackle this problem. By adopting a multi-modal bidirectional encoder and a structure-aware GRU decoder with soft-mask operation, the DSPS model surpass the baseline method by a large margin. All scripts and datasets will be made publicly available at https://github.com/jfma-USTC/HRDoc.