# Finetuning Vision-Language Models as OCR Systems for Low-Resource Languages: A Case Study of Manchu

arXiv: https://arxiv.org/abs/2507.06761 · HTML: https://mlsanity.com/paper/2507.06761/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 9 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 5 of 5)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [An Empirical Study of Scaling Law for OCR](https://mlsanity.com/paper/2401.00028/) (2023)
- [Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model](https://mlsanity.com/paper/2405.09215/) (2024)

## Built on by (most cited first, 5 of 5)

- [OmniOCR: Generalist OCR for Ethnic Minority Languages](https://mlsanity.com/paper/2602.21042/) (2026)
- [DocAtlas: Multilingual Document Understanding Across 80+ Languages](https://mlsanity.com/paper/2605.12623/) (2026)
- [Multi-Expert Routing for Multi-Domain Low-Resource OCR: A Manchu Case Study](https://mlsanity.com/paper/2607.14041/) (2026)
- [Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning](https://mlsanity.com/paper/2604.20813/) (2026)
- [How Far Can Synthetic Data Take Thai OCR?](https://mlsanity.com/paper/2609.03595/) (2026)

## Abstract (arXiv)

Manchu, a critically endangered language essential for understanding early modern Eastern Eurasian history, lacks effective OCR systems that can handle real-world historical documents. This study develops high-performing OCR systems by fine-tuning three open-source vision-language models (LLaMA-3.2-11B, Qwen2.5-VL-7B, Qwen2.5-VL-3B) on 60,000 synthetic Manchu word images using parameter-efficient training. LLaMA-3.2-11B achieved exceptional performance with 98.3% word accuracy and 0.0024 character error rate on synthetic data, while crucially maintaining 93.1% accuracy on real-world handwritten documents. Comparative evaluation reveals substantial advantages over traditional approaches: while a CRNN baseline achieved 99.8% synthetic accuracy, it suffered severe degradation to 72.5% on real documents. Our approach demonstrates effective synthetic-to-real domain transfer, providing a cost-effective solution deployable on accessible infrastructure. This work establishes a transferable framework for endangered language OCR that removes technical and financial barriers in digital humanities, enabling historians and linguists to process historical archives without specialized computing resources. Code and model weights are available at https://github.com/mic7ch1/ManchuAI-OCR.