# When Do VLMs Help Arabic Manuscript OCR? A Cross-Dataset Study

arXiv: https://arxiv.org/abs/2608.22366 · HTML: https://mlsanity.com/paper/2608.22366/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 22 of 22)

- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024)
- [PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culture](https://mlsanity.com/paper/2509.02550/) (2025)
- [KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding](https://mlsanity.com/paper/2502.14949/) (2025)
- [Muharaf: Manuscripts of Handwritten Arabic Dataset for Cursive Text Recognition](https://mlsanity.com/paper/2406.09630/) (2024)
- [BADAM: A Public Dataset for Baseline Detection in Arabic-script Manuscripts](https://mlsanity.com/paper/1907.04041/) (2019)
- [Multimodal LLMs for OCR, OCR Post-Correction, and Named Entity Recognition in Historical Documents](https://mlsanity.com/paper/2504.00414/) (2025)
- [OCR Error Post-Correction with LLMs in Historical Documents: No Free Lunches](https://mlsanity.com/paper/2502.01205/) (2025)
- [Benchmarking Large Language Models for Handwritten Text Recognition](https://mlsanity.com/paper/2503.15195/) (2025)
- [Early evidence of how LLMs outperform traditional systems on OCR/HTR tasks for historical records](https://mlsanity.com/paper/2501.11623/) (2025)
- [Native vs Non-Native Language Prompting: A Comparative Analysis](https://mlsanity.com/paper/2409.07054/) (2024)
- [HATFormer: Historic Handwritten Arabic Text Recognition with Transformers](https://mlsanity.com/paper/2410.02179/) (2024)
- [CHURRO: Making History Readable with an Open-Weight Large Vision-Language Model for High-Accuracy, Low-Cost Historical Text Recognition](https://mlsanity.com/paper/2509.19768/) (2025)
- [From RAG to Agentic RAG for Faithful Islamic Question Answering](https://mlsanity.com/paper/2601.07528/) (2026)
- [Advancements and Challenges in Arabic Optical Character Recognition: A Comprehensive Survey](https://mlsanity.com/paper/2312.11812/) (2023)
- [Fanar 2.0: Arabic Generative AI Stack](https://mlsanity.com/paper/2603.16397/) (2026)
- [Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition](https://mlsanity.com/paper/2407.13559/) (2024)
- [Reference-Based Post-OCR Processing with LLM for Precise Diacritic Text in Historical Document Recognition](https://mlsanity.com/paper/2410.13305/) (2024)
- [SARD: A Large-Scale Synthetic Arabic OCR Dataset for Book-Style Text Recognition](https://mlsanity.com/paper/2505.24600/) (2025)
- [QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation](https://mlsanity.com/paper/2506.02295/) (2025)
- [Evaluating LLMs for Historical Document OCR: A Methodological Framework for Digital Humanities](https://mlsanity.com/paper/2510.06743/) (2025)
- [Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR](https://mlsanity.com/paper/2509.18174/) (2025)

## Abstract (arXiv)

Vision-language models (VLMs) are increasingly being used for document understanding, yet their role in Arabic and Islamic manuscript recognition remains underexplored. To address such a gap in this paper, we evaluate traditional OCR, general-purpose VLMs, Arabic-specialized VLMs, and OCR-conditioned VLM correction across eight Arabic text datasets spanning historical manuscripts, aged printed books, clean print, multi-domain documents, and handwriting. The results show that no single approach dominates across setups. On line-level historical manuscripts, VLMs are close to Tesseract; on page-level manuscript images, they perform better; and in several settings, an OCR-conditioned corrector improves over both standalone OCR and standalone VLMs. The central finding is an OCR-prior recoverability principle: OCR conditioning helps when the OCR output remains visually and textually recoverable, providing anchors that the VLM can refine against the image. It improves recognition on aged print, clean print, mixed-domain Arabic, and some Naskh manuscripts, but degrades performance when the prior is script-mismatched or systematically misleading, as in Maghribi manuscripts and realistic student handwriting. Additional diagnostics show that Arabic VLM-OCR is sensitive to diacritics, preprocessing, generation budget, and repetition loops. These findings support an adaptive OCR-VLM workflow that routes pages according to script, OCR-prior recoverability, length diagnostics, and failure-mode indicators.