# M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis

arXiv: https://arxiv.org/abs/2305.08719 · HTML: https://mlsanity.com/paper/2305.08719/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 21 of 21)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents](https://mlsanity.com/paper/2004.12629/) (2020)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [ISTR: End-to-End Instance Segmentation with Transformers](https://mlsanity.com/paper/2105.00637/) (2021)
- [VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations](https://mlsanity.com/paper/2105.06220/) (2021)
- [SCNet: Training Inference Sample Consistency for Instance Segmentation](https://mlsanity.com/paper/2012.10150/) (2020)
- [dhSegment: A generic deep-learning approach for document segmentation](https://mlsanity.com/paper/1804.10371/) (2018)
- [Cross-Domain Document Object Detection: Benchmark Suite and Method](https://mlsanity.com/paper/2003.13197/) (2020)
- [Marior: Margin Removal and Iterative Content Rectification for Document Dewarping in the Wild](https://mlsanity.com/paper/2207.11515/) (2022)

## Built on by (most cited first, 30 of 46)

- [OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning](https://mlsanity.com/paper/2501.00321/) (2024)
- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)
- [Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want](https://mlsanity.com/paper/2403.20271/) (2024)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm](https://mlsanity.com/paper/2506.05218/) (2025)
- [DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception](https://mlsanity.com/paper/2410.12628/) (2024)
- [Detect Anything via Next Point Prediction](https://mlsanity.com/paper/2510.12798/) (2025)
- [Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanation](https://mlsanity.com/paper/2503.14905/) (2025)
- [dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model](https://mlsanity.com/paper/2512.02498/) (2025)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matching](https://mlsanity.com/paper/2409.03643/) (2024)
- [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](https://mlsanity.com/paper/2605.27365/) (2026)
- [DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks](https://mlsanity.com/paper/2405.04408/) (2024)
- [OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models](https://mlsanity.com/paper/2502.16161/) (2025)
- [RoDLA: Benchmarking the Robustness of Document Layout Analysis Models](https://mlsanity.com/paper/2403.14442/) (2024)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [WordScape: a Pipeline to extract multilingual, visually rich Documents with Layout Annotations from Web Crawl Data](https://mlsanity.com/paper/2312.10188/) (2023)
- [MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios](https://mlsanity.com/paper/2603.28130/) (2026)
- [DocSAM: Unified Document Image Segmentation via Query Decomposition and Heterogeneous Mixed Learning](https://mlsanity.com/paper/2504.04085/) (2025)
- [MonkeyOCRv2: A Visual-Text Foundation Model for Document AI](https://mlsanity.com/paper/2607.11562/) (2026)
- [A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence](https://mlsanity.com/paper/2402.12928/) (2024)
- [PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks](https://mlsanity.com/paper/2503.04065/) (2025)
- [UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis](https://mlsanity.com/paper/2503.15893/) (2025)
- [OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learning](https://mlsanity.com/paper/2510.26213/) (2025)
- [Towards Unified Multi-granularity Text Detection with Interactive Attention](https://mlsanity.com/paper/2405.19765/) (2024)
- [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025)
- [Class-Agnostic Region-of-Interest Matching in Document Images](https://mlsanity.com/paper/2506.21055/) (2025)
- [DREAM: Document Reconstruction via End-to-end Autoregressive Model](https://mlsanity.com/paper/2507.05805/) (2025)
- [KH-FUNSD: A Hierarchical and Fine-Grained Layout Analysis Dataset for Low-Resource Khmer Business Document](https://mlsanity.com/paper/2512.11849/) (2025)
- [Dynamic Relation Transformer for Contextual Text Block Detection](https://mlsanity.com/paper/2401.09232/) (2024)

## Abstract (arXiv)

Document layout analysis is a crucial prerequisite for document understanding, including document retrieval and conversion. Most public datasets currently contain only PDF documents and lack realistic documents. Models trained on these datasets may not generalize well to real-world scenarios. Therefore, this paper introduces a large and diverse document layout analysis dataset called M6Doc. The M6 designation represents six properties: (1) Multi-Format (including scanned, photographed, and PDF documents); (2) Multi-Type (such as scientific articles, textbooks, books, test papers, magazines, newspapers, and notes); (3) Multi-Layout (rectangular, Manhattan, non-Manhattan, and multi-column Manhattan); (4) Multi-Language (Chinese and English); (5) Multi-Annotation Category (74 types of annotation labels with 237,116 annotation instances in 9,080 manually annotated pages); and (6) Modern documents. Additionally, we propose a transformer-based document layout analysis method called TransDLANet, which leverages an adaptive element matching mechanism that enables query embedding to better match ground truth to improve recall, and constructs a segmentation branch for more precise document image instance segmentation. We conduct a comprehensive evaluation of M6Doc with various layout analysis methods and demonstrate its effectiveness. TransDLANet achieves state-of-the-art performance on M6Doc with 64.5% mAP. The M6Doc dataset will be available at https://github.com/HCIILAB/M6Doc.