mlsanityData to 7 Oct 2026

Exploring OCR-augmented Generation for Bilingual VQA

arXiv 2510.02543 ↗First posted 2 Oct 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We investigate OCR-augmented generation with Vision Language Models (VLMs), exploring tasks in Korean and English toward multilingualism. To support research in this domain, we train and release KLOCR, a strong bilingual OCR baseline trained on 100M instances to augment VLMs with OCR ability. To complement existing VQA benchmarks, we curate KOCRBench for Korean VQA, and analyze different prompting methods. Extensive experiments show that OCR-extracted text significantly boosts performance across open source and commercial models. Our work offers new insights into OCR-augmented generation for bilingual VQA. Model, code, and data are available at https://github.com/JHLee0513/KLOCR.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 60 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

35 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Models4 Dec

Other references · 34

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2019Towards VQA Models That Can Readcited by 1,696
  7. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  9. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  10. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
  11. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  12. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
22 more
  1. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 278
  2. 2019Character Region Awareness for Text Detectioncited by 273
  3. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 254
  4. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 240
  5. 2019Real-time Scene Text Detection with Differentiable Binarizationcited by 212
  6. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 174
  7. 2020PP-OCR: A Practical Ultra Lightweight OCR Systemcited by 147
  8. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  9. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 122
  10. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  11. 2022PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR Systemcited by 105
  12. 2022Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusioncited by 94
  13. 2022SVTR: Scene Text Recognition with a Single Visual Modelcited by 86
  14. 2021LayoutReader: Pre-training of Text and Layout for Reading Order Detectioncited by 79
  15. 2021Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solutioncited by 64
  16. 2022DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformercited by 50
  17. 2023Vision Grid Transformer for Document Layout Analysiscited by 45
  18. 2021SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Modelscited by 43
  19. 2024DocVLM: Make Your VLM an Efficient Readercited by 27
  20. 2023Modeling Entities as Semantic Points for Visual Information Extraction in the Wildcited by 20
  21. 2023An Empirical Study of Scaling Law for OCRcited by 7
  22. 2023High-Performance Transformers for Table Structure Recognition Need Early Convolutionscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026KorMedMCQA-V: A Multimodal Benchmark for Evaluating Vision-Language Models on the Korean Medical Licensing Examinationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025VARCO-VISION-2.0 Technical Report
  2. 2024VARCO-VISION: Expanding Frontiers in Korean Vision-Language Models
  3. 2024TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models
  4. 2025Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning
  5. 2026Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models
  6. 2024Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models
  7. 2026Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models
  8. 2025Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation