# UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents arXiv: https://arxiv.org/abs/2602.07038 ยท HTML: https://mlsanity.com/paper/2602.07038/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 35 of its 58 references that are in the map. - First posted on arXiv 3 Feb 2026. ## Placed in - Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/) ## Builds on (most cited first, 30 of 35) - [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025) - [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019) - [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022) - [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024) - [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024) - [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019) - [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021) - [SmolVLM: Redefining small and efficient multimodal models](https://mlsanity.com/paper/2504.05299/) (2025) - [OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning](https://mlsanity.com/paper/2501.00321/) (2024) - [DocLLM: A layout-aware generative language model for multimodal document understanding](https://mlsanity.com/paper/2401.00908/) (2023) - [mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding](https://mlsanity.com/paper/2409.03420/) (2024) - [CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy](https://mlsanity.com/paper/2412.02210/) (2024) - [PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks](https://mlsanity.com/paper/2004.07464/) (2020) - [Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solution](https://mlsanity.com/paper/2102.06732/) (2021) - [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022) - [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023) - [Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution](https://mlsanity.com/paper/2305.07498/) (2023) - [Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation](https://mlsanity.com/paper/2310.16809/) (2023) - [HRVDA: High-Resolution Visual Document Assistant](https://mlsanity.com/paper/2404.06918/) (2024) - [DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming](https://mlsanity.com/paper/2406.19101/) (2024) - [CloudScan - A configuration-free invoice analysis system using recurrent neural networks](https://mlsanity.com/paper/1708.07403/) (2017) - [DocVLM: Make Your VLM an Efficient Reader](https://mlsanity.com/paper/2412.08746/) (2024) - [Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction](https://mlsanity.com/paper/2310.11016/) (2023) - [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024) - [Modeling Entities as Semantic Points for Visual Information Extraction in the Wild](https://mlsanity.com/paper/2303.13095/) (2023) - [DocParser: End-to-end OCR-free Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2304.12484/) (2023) - [Hierarchical Visual Feature Aggregation for OCR-Free Document Understanding](https://mlsanity.com/paper/2411.05254/) (2024) - [Business Document Information Extraction: Towards Practical Benchmarks](https://mlsanity.com/paper/2206.11229/) (2022) - [Test-Time Adaptation for Visual Document Understanding](https://mlsanity.com/paper/2206.07240/) (2022) - [TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/2207.06744/) (2022) ## Built on by (most cited first, 5 of 5) - [Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis](https://mlsanity.com/paper/2607.04636/) (2026) - [ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction](https://mlsanity.com/paper/2607.29677/) (2026) - [Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction](https://mlsanity.com/paper/2608.01792/) (2026) - [DocMIDE: Learning Multi-Hop Implicit Derivation in Visually Rich Documents](https://mlsanity.com/paper/2609.24092/) (2026) - [EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI](https://mlsanity.com/paper/2610.16689/) (2026) ## Abstract (arXiv) Key Information Extraction (KIE) from real-world documents remains challenging due to substantial variations in layout structures, visual quality, and task-specific information requirements. Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images. To enable a comprehensive and systematic evaluation across realistic and diverse application scenarios, we introduce UNIKIE-BENCH, a unified benchmark designed to rigorously evaluate the KIE capabilities of LMMs. UNIKIE-BENCH consists of two complementary tracks: a constrained-category KIE track with scenario-predefined schemas that reflect practical application needs, and an open-category KIE track that extracts any key information that is explicitly present in the document. Experiments on 15 state-of-the-art LMMs reveal substantial performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios. These findings underscore persistent challenges in grounding accuracy and layout-aware reasoning for LMM-based KIE. All codes and datasets are available at https://github.com/NEUIR/UNIKIE-BENCH.