mlsanityData to 14 Oct 2026

What Can Low Resource Languages Learn From Each Other?

arXiv 2608.27753 ↗First posted 27 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 25 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

18 of 25 references.

References · 18

  1. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  2. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  3. 2022Editing Models with Task Arithmeticcited by 1,120
  4. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  5. 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 387
  6. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 279
  7. 2022Scene Text Recognition with Permuted Autoregressive Sequence Modelscited by 149
  8. 2019ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019cited by 139
  9. 2023Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Maskscited by 133
  10. 2019ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboardcited by 128
  11. 2021SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Modelscited by 43
  12. 2021Benchmarking Scene Text Recognition in Devanagari, Telugu and Malayalamcited by 11
6 more
  1. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMscited by 8
  2. 2022Transfer Learning for Scene Text Recognition in Indian Languagescited by 6
  3. 2023Cross-Lingual Learning in Multilingual Scene Text Recognitioncited by 3
  4. 2022Towards Deployable OCR models for Indic languagescited by 2
  5. 2024Spanish TrOCR: Leveraging Transfer Learning for Language Adaptationcited by 2
  6. 2024IndicSTR12: A Dataset for Indic Scene Text Recognitioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Models
  2. 2025Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning
  3. 2026Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models
  4. 2025Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model
  5. 2026GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
  6. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Survey
  7. 2023Joint Adaptive Representations for Image-Language Learning
  8. 2026Language-Conditioned Visual Grounding with CLIP Multilingual