# Beyond Document Page Classification: Design, Datasets, and Challenges

arXiv: https://arxiv.org/abs/2308.12896 · HTML: https://mlsanity.com/paper/2308.12896/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 65 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [InfographicVQA](https://mlsanity.com/paper/2104.12756/) (2021)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval](https://mlsanity.com/paper/1502.07058/) (2015)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [DiT: Self-supervised Pre-training for Document Image Transformer](https://mlsanity.com/paper/2203.02378/) (2022)
- [DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis](https://mlsanity.com/paper/2206.01062/) (2022)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [Unifying Vision, Text, and Layout for Universal Document Processing](https://mlsanity.com/paper/2212.02623/) (2022)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [SelfDoc: Self-Supervised Document Representation Learning](https://mlsanity.com/paper/2106.03331/) (2021)
- [Hierarchical multimodal transformers for Multi-Page DocVQA](https://mlsanity.com/paper/2212.05935/) (2022)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [Unified Pretraining Framework for Document Understanding](https://mlsanity.com/paper/2204.10939/) (2022)
- [Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layouts](https://mlsanity.com/paper/2105.05796/) (2021)
- [Towards Complex Document Understanding By Discrete Reasoning](https://mlsanity.com/paper/2207.11871/) (2022)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification](https://mlsanity.com/paper/2211.15259/) (2022)

## Built on by (most cited first, 10 of 10)

- [Overcoming Common Flaws in the Evaluation of Selective Classification Systems](https://mlsanity.com/paper/2407.01032/) (2024)
- [μgat: Improving Single-Page Document Parsing by Providing Multi-Page Context](https://mlsanity.com/paper/2408.15646/) (2024)
- [Large Language Models for Page Stream Segmentation](https://mlsanity.com/paper/2408.11981/) (2024)
- [DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splitting](https://mlsanity.com/paper/2602.15958/) (2026)
- [Diving into the Depths of Spotting Text in Multi-Domain Noisy Scenes](https://mlsanity.com/paper/2310.00558/) (2023)
- [KH-FUNSD: A Hierarchical and Fine-Grained Layout Analysis Dataset for Low-Resource Khmer Business Document](https://mlsanity.com/paper/2512.11849/) (2025)
- [GlobalDoc: A Cross-Modal Vision-Language Framework for Real-World Document Image Retrieval and Classification](https://mlsanity.com/paper/2309.05756/) (2023)
- [DistilDoc: Knowledge Distillation for Visually-Rich Document Applications](https://mlsanity.com/paper/2406.08226/) (2024)
- [ECUASn: A family of metrics for principled evaluation of uncertainty-augmented systems](https://mlsanity.com/paper/2605.20490/) (2026)
- [Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms](https://mlsanity.com/paper/2607.21780/) (2026)

## Abstract (arXiv)

This paper highlights the need to bring document classification benchmarking closer to real-world applications, both in the nature of data tested (X: multi-channel, multi-paged, multi-industry; Y: class distributions and label set variety) and in classification tasks considered (f: multi-page document, page stream, and document bundle classification, ...). We identify the lack of public multi-page document classification datasets, formalize different classification tasks arising in application scenarios, and motivate the value of targeting efficient multi-page document representations. An experimental study on proposed multi-page document classification datasets demonstrates that current benchmarks have become irrelevant and need to be updated to evaluate complete documents, as they naturally occur in practice. This reality check also calls for more mature evaluation methodologies, covering calibration evaluation, inference complexity (time-memory), and a range of realistic distribution shifts (e.g., born-digital vs. scanning noise, shifting page order). Our study ends on a hopeful note by recommending concrete avenues for future improvements.