mlsanityData to 7 Oct 2026

Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition

arXiv 2407.13559 ↗First posted 18 Jul 2024cs.CV cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Arabic Optical Character Recognition (OCR) and Handwriting Recognition (HWR) pose unique challenges due to the cursive and context-sensitive nature of the Arabic script. This study introduces Qalam, a novel foundation model designed for Arabic OCR and HWR, built on a SwinV2 encoder and RoBERTa decoder architecture. Our model significantly outperforms existing methods, achieving a Word Error Rate (WER) of just 0.80% in HWR tasks and 1.18% in OCR tasks. We train Qalam on a diverse dataset, including over 4.5 million images from Arabic manuscripts and a synthetic dataset comprising 60k image-text pairs. Notably, Qalam demonstrates exceptional handling of Arabic diacritics, a critical feature in Arabic scripts. Furthermore, it shows a remarkable ability to process high-resolution inputs, addressing a common limitation in current OCR systems. These advancements underscore Qalam's potential as a leading solution for Arabic script recognition, offering a significant leap in accuracy and efficiency.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 30 of its 74 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

30 of 74 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  12. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
18 more
  1. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  2. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  3. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
  4. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 278
  5. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
  6. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  7. 2016Recursive Recurrent Nets with Attention Modeling for OCR in the Wildcited by 74
  8. 2020Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognitioncited by 71
  9. 2019Evaluating Sequence-to-Sequence Models for Handwritten Text Recognitioncited by 58
  10. 2020End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Networkcited by 50
  11. 2023Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluationcited by 46
  12. 2016Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attentioncited by 45
  13. 2013Dropout improves Recurrent Neural Networks for Handwriting Recognitioncited by 39
  14. 2024Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarkscited by 32
  15. 2018Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networkscited by 31
  16. 2020Recurrence-free unconstrained handwritten text recognition using gated fully convolutional networkcited by 23
  17. 2021Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixescited by 18
  18. 2014AlexU-Word: A New Dataset for Isolated-Word Closed-Vocabulary Offline Arabic Handwriting Recognitioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptationcited by 5
  2. 2025Large Multimodal Models for Low-Resource Languages: A Surveycited by 4
  3. 2025Arabic Multimodal Machine Learning: Datasets, Applications, Approaches, and Challengescited by 1
  4. 2025Invizo: Arabic Handwritten Document Optical Character Recognition Solutioncited by 0
  5. 2025A-SEA3L-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generationcited by 0
  6. 2026LLM-Driven AutoML for Cross-Lingual Handwritten OCR: Closed-Loop Neural Architecture Search with GPT-5, GPT-4o, and Claude Sonnet 4cited by 0
  7. 2026When Do VLMs Help Arabic Manuscript OCR? A Cross-Dataset Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SARD: A Large-Scale Synthetic Arabic OCR Dataset for Book-Style Text Recognition
  2. 2026AtlasOCR: Building the First Open-Source Darija OCR Model with Vision Language Models
  3. 2026RefLAM: A Reference-Grounded Line Annotation Pipeline for Historical Arabic Manuscripts
  4. 2025KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding
  5. 2025Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR
  6. 2025PsOCR: Benchmarking Large Multimodal Models for Optical Character Recognition in Low-resource Pashto Language
  7. 2025E-ARMOR: Edge case Assessment and Review of Multilingual Optical Character Recognition
  8. 2026KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR