# Multi-domain Multi-modal Document Classification Benchmark with a Multi-level Taxonomy

arXiv: https://arxiv.org/abs/2605.10550 · HTML: https://mlsanity.com/paper/2605.10550/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 11 May 2026.

## Builds on (most cited first, 13 of 13)

- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis](https://mlsanity.com/paper/2206.01062/) (2022)
- [HDLTex: Hierarchical Deep Learning for Text Classification](https://mlsanity.com/paper/1709.08267/) (2017)
- [TableBank: A Benchmark Dataset for Table Detection and Recognition](https://mlsanity.com/paper/1903.01949/) (2019)
- [SECTOR: A Neural Model for Coherent Topic Segmentation and Classification](https://mlsanity.com/paper/1902.04793/) (2019)
- [ChatGPT-Powered Hierarchical Comparisons for Image Classification](https://mlsanity.com/paper/2311.00206/) (2023)
- [Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding](https://mlsanity.com/paper/2212.09621/) (2022)
- [LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understanding](https://mlsanity.com/paper/2305.18721/) (2023)
- [Evaluating Out-of-Distribution Performance on Document Image Classifiers](https://mlsanity.com/paper/2210.07448/) (2022)
- [Hierarchical Selective Classification](https://mlsanity.com/paper/2405.11533/) (2024)
- [Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels](https://mlsanity.com/paper/2509.16596/) (2025)
- [DocumentNet: Bridging the Data Gap in Document Pre-Training](https://mlsanity.com/paper/2306.08937/) (2023)
- [An Independent Safety Evaluation of Kimi K2.5](https://mlsanity.com/paper/2604.03121/) (2026)

## Abstract (arXiv)

Document classification forms the backbone of modern enterprise content management, yet existing benchmarks remain trapped in oversimplified paradigms -- single domain settings with flat label structures -- that bear little resemblance to the hierarchical, multi-modal, and cross-domain nature of real-world business documents. This gap not only misrepresents practical complexity but also stifles progress toward industrially viable document intelligence. To bridge this gap, we construct the first Multi-level, Multi-domain, Multi-modal document classification Benchmark (MMM-Bench). MMM-Bench includes (1) a deeply hierarchical taxonomy spanning five levels that capture the authentic organizational logic of business documentation; and (2) 5,990 real-world multi-modal documents meticulously curated from 12 commercial domains in Alibaba. Each document is manually annotated with a complete hierarchical path by domain experts. We establish comprehensive baselines on MMM-Bench, which consists of open-weight models and API-based models. Through systematic experiments, we identify four fundamental challenges within MMM-Bench and propose corresponding insights. To provide a solid foundation for advancing research in multi-level, multi-domain document classification, we release all of the data and the evaluation toolkit of MMM-Bench at https://github.com/MMMDC-Bench/MMMDC-Bench.