# Towards End-to-End In-Image Neural Machine Translation

arXiv: https://arxiv.org/abs/2010.10648 · HTML: https://mlsanity.com/paper/2010.10648/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 20 of 20)

- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Language Modelling with Pixels](https://mlsanity.com/paper/2207.06991/) (2022)
- [CLIPPO: Image-and-Language Understanding from Pixels Only](https://mlsanity.com/paper/2212.08045/) (2022)
- [Robust Open-Vocabulary Translation from Visual Text Representations](https://mlsanity.com/paper/2104.08211/) (2021)
- [Exploring Better Text Image Translation with Multimodal Codebook](https://mlsanity.com/paper/2305.17415/) (2023)
- [Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task](https://mlsanity.com/paper/2210.03887/) (2022)
- [MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translation](https://mlsanity.com/paper/2412.07147/) (2024)
- [AnyTrans: Translate AnyText in the Image with Large Scale Models](https://mlsanity.com/paper/2406.11432/) (2024)
- [Exploring In-Image Machine Translation with Real-World Background](https://mlsanity.com/paper/2505.15282/) (2025)
- [PRIM: Towards Practical In-Image Multilingual Machine Translation](https://mlsanity.com/paper/2509.05146/) (2025)
- [A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges](https://mlsanity.com/paper/2405.12669/) (2024)
- [E2TIMT: Efficient and Effective Modal Adapter for Text Image Machine Translation](https://mlsanity.com/paper/2305.05166/) (2023)
- [Show Me the World in My Language: Establishing the First Baseline for Scene-Text to Scene-Text Translation](https://mlsanity.com/paper/2308.03024/) (2023)
- [Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation](https://mlsanity.com/paper/2602.21956/) (2026)
- [Multi-Teacher Knowledge Distillation For Text Image Machine Translation](https://mlsanity.com/paper/2305.05226/) (2023)
- [PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models](https://mlsanity.com/paper/2509.12278/) (2025)
- [MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation](https://mlsanity.com/paper/2604.16943/) (2026)
- [LLaVA-NeuMT: Selective Layer-Neuron Modulation for Efficient Multilingual Multimodal Translation](https://mlsanity.com/paper/2507.18940/) (2025)
- [Vectra: A New Metric, Dataset, and Model for Visual Quality Assessment in E-Commerce In-Image Machine Translation](https://mlsanity.com/paper/2602.07014/) (2026)
- [VaaWIT: Visual-Aware Adaptation of Large Language Models for Multilingual Web Image Translation](https://mlsanity.com/paper/2605.24675/) (2026)

## Abstract (arXiv)

In this paper, we offer a preliminary investigation into the task of in-image machine translation: transforming an image containing text in one language into an image containing the same text in another language. We propose an end-to-end neural model for this task inspired by recent approaches to neural machine translation, and demonstrate promising initial results based purely on pixel-level supervision. We then offer a quantitative and qualitative evaluation of our system outputs and discuss some common failure modes. Finally, we conclude with directions for future work.