# KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding

arXiv: https://arxiv.org/abs/2502.14949 · HTML: https://mlsanity.com/paper/2502.14949/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning](https://mlsanity.com/paper/2402.12185/) (2024)
- [TableFormer: Table Structure Understanding with Transformers](https://mlsanity.com/paper/2203.01017/) (2022)
- [Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks](https://mlsanity.com/paper/2403.01031/) (2024)
- [Muharaf: Manuscripts of Handwritten Arabic Dataset for Cursive Text Recognition](https://mlsanity.com/paper/2406.09630/) (2024)

## Built on by (most cited first, 15 of 15)

- [Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence](https://mlsanity.com/paper/2606.15932/) (2026)
- [Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR](https://mlsanity.com/paper/2509.18174/) (2025)
- [POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering](https://mlsanity.com/paper/2507.11939/) (2025)
- [A Survey of OCR Evaluation Methods and Metrics and the Invisibility of Historical Documents](https://mlsanity.com/paper/2603.25761/) (2026)
- [GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts](https://mlsanity.com/paper/2604.12978/) (2026)
- [Cross-Lingual SynthDocs: A Large-Scale Synthetic Corpus for Any to Arabic OCR and Document Understanding](https://mlsanity.com/paper/2511.04699/) (2025)
- [TexOCR: Advancing Document OCR Models for Compilable Page-to-LaTeX Reconstruction](https://mlsanity.com/paper/2604.22880/) (2026)
- [DocAtlas: Multilingual Document Understanding Across 80+ Languages](https://mlsanity.com/paper/2605.12623/) (2026)
- [PsOCR: Benchmarking Large Multimodal Models for Optical Character Recognition in Low-resource Pashto Language](https://mlsanity.com/paper/2505.10055/) (2025)
- [A-SEA3L-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation](https://mlsanity.com/paper/2509.02864/) (2025)
- [Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks](https://mlsanity.com/paper/2510.06071/) (2025)
- [Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Application](https://mlsanity.com/paper/2608.18289/) (2026)
- [Hadith computational science in the age of large language models: a critical narrative review](https://mlsanity.com/paper/2608.20364/) (2026)
- [When Do VLMs Help Arabic Manuscript OCR? A Cross-Dataset Study](https://mlsanity.com/paper/2608.22366/) (2026)
- [AraMS-28k: The Largest Publicly Released Line-Level Dataset of Historical Arabic Manuscripts with Margin and Insertion-Anchor Annotations](https://mlsanity.com/paper/2608.26921/) (2026)

## Abstract (arXiv)

With the growing adoption of Retrieval-Augmented Generation (RAG) in document processing, robust text recognition has become increasingly critical for knowledge extraction. While OCR (Optical Character Recognition) for English and other languages benefits from large datasets and well-established benchmarks, Arabic OCR faces unique challenges due to its cursive script, right-to-left text flow, and complex typographic and calligraphic features. We present KITAB-Bench, a comprehensive Arabic OCR benchmark that fills the gaps in current evaluation systems. Our benchmark comprises 8,809 samples across 9 major domains and 36 sub-domains, encompassing diverse document types including handwritten text, structured tables, and specialized coverage of 21 chart types for business intelligence. Our findings show that modern vision-language models (such as GPT-4o, Gemini, and Qwen) outperform traditional OCR approaches (like EasyOCR, PaddleOCR, and Surya) by an average of 60% in Character Error Rate (CER). Furthermore, we highlight significant limitations of current Arabic OCR models, particularly in PDF-to-Markdown conversion, where the best model Gemini-2.0-Flash achieves only 65% accuracy. This underscores the challenges in accurately recognizing Arabic text, including issues with complex fonts, numeral recognition errors, word elongation, and table structure detection. This work establishes a rigorous evaluation framework that can drive improvements in Arabic document analysis methods and bridge the performance gap with English OCR technologies.