# BanglaWild: An In-the-Wild Bengali Scene Text Recognition Benchmark for OCR and Vision-Language Models

arXiv: https://arxiv.org/abs/2608.03884 · HTML: https://mlsanity.com/paper/2608.03884/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 14 of 14)

- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Self-Preference Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2410.21819/) (2024)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [Preference Leakage: A Contamination Problem in LLM-as-a-judge](https://mlsanity.com/paper/2502.01534/) (2025)
- [Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2508.06709/) (2025)
- [IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs](https://mlsanity.com/paper/2511.04727/) (2025)
- [Transfer Learning for Scene Text Recognition in Indian Languages](https://mlsanity.com/paper/2201.03180/) (2022)
- [ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai](https://mlsanity.com/paper/2511.04479/) (2025)
- [Deciphering the Underserved: Benchmarking LLM OCR for Low-Resource Scripts](https://mlsanity.com/paper/2412.16119/) (2024)
- [When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCR](https://mlsanity.com/paper/2604.22774/) (2026)
- [BN-HTRd: A Benchmark Dataset for Document Level Offline Bangla Handwritten Text Recognition (HTR) and Line Segmentation](https://mlsanity.com/paper/2206.08977/) (2022)
- [Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understanding](https://mlsanity.com/paper/2511.23071/) (2025)

## Abstract (arXiv)

In-the-wild Bengali scene text recognition is largely unmeasured: existing resources target handwritten documents or constrained sign-board parsing, report only aggregate edit-distance metrics, and evaluate either conventional OCR or VLMs, never both on the same in-the-wild data. To address this gap, we introduce BANGLAWILD, a benchmark of 2,535 Bengali scene text images, each paired with a verbatim gold transcription, two categorical axes, four diagnostic attributes, and an orthographically standard form where the in-image text deviates from canonical spelling. We evaluate fifteen VLMs and three conventional OCR systems under three prompting strategies, fine-tune 6 open-source models with LoRA, and complement edit-distance metrics with an LLM-as-a-Judge evaluation. Our results reveal a persistent gap in which larger models within the same family do not outperform smaller ones. Our fifteen-class error taxonomy shows that visual mis-recognition accounts for ~60% of errors in the strongest systems, while conjunct-related errors contribute under 2%, challenging a long-standing assumption in Bengali OCR research; the same visual dominant profile also holds across architectures, including the one conventional baseline that reads Bengali reliably. Prompt language mainly affects cross-script drift and LoRA reduces catastrophic failures in weak models without lifting the ceiling on already competent ones. Code and data will be publicly released.