Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation
Not on the map. Why?
Abstract (arXiv)
Text Image Machine Translation (TIMT) aims to translate text embedded in images in the source-language into target-language, requiring synergistic integration of visual perception and linguistic understanding. Existing TIMT methods, whether cascaded pipelines or end-to-end multimodal large language models (MLLMs),struggle with high-resolution text-rich images due to cluttered layouts, diverse fonts, and non-textual distractions, resulting in text omission, semantic drift, and contextual inconsistency. To address these challenges, we propose GLoTran, a global-local dual visual perception framework for MLLM-based TIMT. GLoTran integrates a low-resolution global image with multi-scale region-level text image slices under an instruction-guided alignment strategy, conditioning MLLMs to maintain scene-level contextual consistency while faithfully capturing fine-grained textual details. Moreover, to realize this dual-perception paradigm, we construct GLoD, a large-scale text-rich TIMT dataset comprising 510K high-resolution global-local image-text pairs covering diverse real-world scenarios. Extensive experiments demonstrate that GLoTran substantially improves translation completeness and accuracy over state-of-the-art MLLMs, offering a new paradigm for fine-grained TIMT under high-resolution and text-rich conditions.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 25 Feb 2026.
Builds on
47 of 67 references.
References · 47
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
- 2025Qwen3 Technical Reportcited by 8,510
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,986
- 2025Qwen2.5-VL Technical Reportcited by 5,499
- 2024GPT-4o System Cardcited by 4,536
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,191
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,546
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,170
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,798
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,027
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 726
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 706
35 more
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 653
- 2020Language-agnostic BERT Sentence Embeddingcited by 622
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 470
- 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
- 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 289
- 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 258
- 2023Nougat: Neural Optical Understanding for Academic Documentscited by 242
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 153
- 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
- 2025PaddleOCR 3.0 Technical Reportcited by 138
- 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
- 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 101
- 2025Scalable Vision Language Model Training via High Quality Data Curationcited by 55
- 2024Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillationcited by 46
- 2024TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understandingcited by 37
- 2024Language Imbalance Driven Rewarding for Multilingual Self-improvingcited by 27
- 2023Exploring Better Text Image Translation with Multimodal Codebookcited by 23
- 2020Towards Fully Automated Manga Translationcited by 21
- 2024LOVA3: Learning to Visual Question Answering, Asking and Assessmentcited by 21
- 2020Towards End-to-End In-Image Neural Machine Translationcited by 20
- 2022Improving End-to-End Text Image Translation From the Auxiliary Text Translation Taskcited by 20
- 2024Translatotron-V(ison): An End-to-End Model for In-Image Machine Translationcited by 14
- 2024Hierarchical Visual Feature Aggregation for OCR-Free Document Understandingcited by 12
- 2024MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translationcited by 12
- 2025Recognition-Synergistic Scene Text Editingcited by 11
- 2025FlashBias: Fast Computation of Attention with Biascited by 11
- 2024AnyTrans: Translate AnyText in the Image with Large Scale Modelscited by 8
- 2024Paying More Attention to Source Context: Mitigating Unfaithful Translations from Large Language Modelcited by 6
- 2025Qianfan-VL: Domain-Enhanced Universal Vision-Language Modelscited by 6
- 2024Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Modelscited by 5
- 2025From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selectioncited by 5
- 2025MT3: Scaling MLLM-based Text Image Machine Translation via Multi-Task Reinforcement Learningcited by 5
- 2025Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiencycited by 5
- 2025Describe Anything Model for Visual Question Answering on Text-rich Imagescited by 5
- 2025Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translationcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation
- 2025Dual-branch Prompting for Multimodal Machine Translation
- 2026Comparative Evaluation of Machine Translation Systems on Images with Text
- 2024INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model
- 2025Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation
- 2024Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
- 2025Multimodal Language Models See Better When They Look Shallower
- 2024The Revolution of Multimodal Large Language Models: A Survey