mlsanityData to 14 Oct 2026

BanglaWild: An In-the-Wild Bengali Scene Text Recognition Benchmark for OCR and Vision-Language Models

arXiv 2608.03884 ↗First posted 4 Aug 2026cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

In-the-wild Bengali scene text recognition is largely unmeasured: existing resources target handwritten documents or constrained sign-board parsing, report only aggregate edit-distance metrics, and evaluate either conventional OCR or VLMs, never both on the same in-the-wild data. To address this gap, we introduce BANGLAWILD, a benchmark of 2,535 Bengali scene text images, each paired with a verbatim gold transcription, two categorical axes, four diagnostic attributes, and an orthographically standard form where the in-image text deviates from canonical spelling. We evaluate fifteen VLMs and three conventional OCR systems under three prompting strategies, fine-tune 6 open-source models with LoRA, and complement edit-distance metrics with an LLM-as-a-Judge evaluation. Our results reveal a persistent gap in which larger models within the same family do not outperform smaller ones. Our fifteen-class error taxonomy shows that visual mis-recognition accounts for ~60% of errors in the strongest systems, while conjunct-related errors contribute under 2%, challenging a long-standing assumption in Bengali OCR research; the same visual dominant profile also holds across architectures, including the one conventional baseline that reads Bengali reliably. Prompt language mainly affects cross-script drift and LoRA reduces catastrophic failures in weak models without lifting the ceiling on already competent ones. Code and data will be publicly released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 31 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

14 of 31 references.

References · 14

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  2. 2018Datasheets for Datasetscited by 1,386
  3. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  4. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  5. 2019ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019cited by 139
  6. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 125
  7. 2025Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judgecited by 36
  8. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMscited by 8
  9. 2022Transfer Learning for Scene Text Recognition in Indian Languagescited by 6
  10. 2025ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thaicited by 5
  11. 2024Deciphering the Underserved: Benchmarking LLM OCR for Low-Resource Scriptscited by 4
  12. 2026When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCRcited by 4
2 more
  1. 2022BN-HTRd: A Benchmark Dataset for Document Level Offline Bangla Handwritten Text Recognition (HTR) and Line Segmentationcited by 3
  2. 2025Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understandingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning
  2. 2026Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems
  3. 2026PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks
  4. 2026KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR
  5. 2025PsOCR: Benchmarking Large Multimodal Models for Optical Character Recognition in Low-resource Pashto Language
  6. 2025Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning
  7. 2026JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding
  8. 2026Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects