# DSG: An End-to-End Document Structure Generator

arXiv: https://arxiv.org/abs/2310.09118 · HTML: https://mlsanity.com/paper/2310.09118/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Learning to Compose Dynamic Tree Structures for Visual Contexts](https://mlsanity.com/paper/1812.01880/) (2018)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [DiT: Self-supervised Pre-training for Document Image Transformer](https://mlsanity.com/paper/2203.02378/) (2022)
- [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://mlsanity.com/paper/2110.00061/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Unified Pretraining Framework for Document Understanding](https://mlsanity.com/paper/2204.10939/) (2022)
- [Segmentation-grounded Scene Graph Generation](https://mlsanity.com/paper/2104.14207/) (2021)
- [HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures](https://mlsanity.com/paper/2303.13839/) (2023)
- [Fonduer: Knowledge Base Construction from Richly Formatted Data](https://mlsanity.com/paper/1703.05028/) (2017)
- [DocBed: A Multi-Stage OCR Solution for Documents with Complex Layouts](https://mlsanity.com/paper/2202.01414/) (2022)

## Built on by (most cited first, 7 of 7)

- [DLAFormer: An End-to-End Transformer For Document Layout Analysis](https://mlsanity.com/paper/2405.11757/) (2024)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis](https://mlsanity.com/paper/2503.15893/) (2025)
- [EDocNet: Efficient Datasheet Layout Analysis Based on Focus and Global Knowledge Distillation](https://mlsanity.com/paper/2502.16541/) (2025)
- [M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models](https://mlsanity.com/paper/2605.18774/) (2026)
- [DOSA: A Tree-Guided, Self-Regressive Framework for Long Document Structure Analysis](https://mlsanity.com/paper/2607.22679/) (2026)
- [Heading Hierarchy Reconstruction for Complex and Long Documents with Level-Wise Multimodal Memory](https://mlsanity.com/paper/2610.18238/) (2026)

## Abstract (arXiv)

Information in industry, research, and the public sector is widely stored as rendered documents (e.g., PDF files, scans). Hence, to enable downstream tasks, systems are needed that map rendered documents onto a structured hierarchical format. However, existing systems for this task are limited by heuristics and are not end-to-end trainable. In this work, we introduce the Document Structure Generator (DSG), a novel system for document parsing that is fully end-to-end trainable. DSG combines a deep neural network for parsing (i) entities in documents (e.g., figures, text blocks, headers, etc.) and (ii) relations that capture the sequence and nested structure between entities. Unlike existing systems that rely on heuristics, our DSG is trained end-to-end, making it effective and flexible for real-world applications. We further contribute a new, large-scale dataset called E-Periodica comprising real-world magazines with complex document structures for evaluation. Our results demonstrate that our DSG outperforms commercial OCR tools and, on top of that, achieves state-of-the-art performance. To the best of our knowledge, our DSG system is the first end-to-end trainable system for hierarchical document parsing.