# Towards Fully Automated Manga Translation

arXiv: https://arxiv.org/abs/2012.14271 · HTML: https://mlsanity.com/paper/2012.14271/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2020.

## Built on by (most cited first, 22 of 22)

- [The Manga Whisperer: Automatically Generating Transcriptions for Comics](https://mlsanity.com/paper/2401.10224/) (2024)
- [Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task](https://mlsanity.com/paper/2210.03887/) (2022)
- [CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding](https://mlsanity.com/paper/2407.03550/) (2024)
- [MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translation](https://mlsanity.com/paper/2412.07147/) (2024)
- [Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Names](https://mlsanity.com/paper/2408.00298/) (2024)
- [Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering](https://mlsanity.com/paper/2204.04621/) (2022)
- [Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion](https://mlsanity.com/paper/2404.13993/) (2024)
- [MangaUB: A Manga Understanding Benchmark for Large Multimodal Models](https://mlsanity.com/paper/2407.19034/) (2024)
- [M2C: Towards Automatic Multimodal Manga Complement](https://mlsanity.com/paper/2310.17130/) (2023)
- [A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges](https://mlsanity.com/paper/2405.12669/) (2024)
- [E2TIMT: Efficient and Effective Modal Adapter for Text Image Machine Translation](https://mlsanity.com/paper/2305.05166/) (2023)
- [MT3: Scaling MLLM-based Text Image Machine Translation via Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2505.19714/) (2025)
- [Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation](https://mlsanity.com/paper/2507.07572/) (2025)
- [MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation](https://mlsanity.com/paper/2603.23896/) (2026)
- [Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation](https://mlsanity.com/paper/2602.21956/) (2026)
- [Multi-Teacher Knowledge Distillation For Text Image Machine Translation](https://mlsanity.com/paper/2305.05226/) (2023)
- [Oracle Bone Script Similiar Character Screening Approach Based on Simsiam Contrastive Learning and Supervised Learning](https://mlsanity.com/paper/2408.06811/) (2024)
- [Context-Informed Machine Translation of Manga using Multimodal Large Language Models](https://mlsanity.com/paper/2411.02589/) (2024)
- [Manga109-v2026: Revisiting Manga109 Annotations for Modern Manga Understanding](https://mlsanity.com/paper/2605.21182/) (2026)
- [Detection of Furigana Text in Images](https://mlsanity.com/paper/2207.03960/) (2022)
- [LLaVA-NeuMT: Selective Layer-Neuron Modulation for Efficient Multilingual Multimodal Translation](https://mlsanity.com/paper/2507.18940/) (2025)
- [Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga](https://mlsanity.com/paper/2610.13840/) (2026)

## Abstract (arXiv)

We tackle the problem of machine translation of manga, Japanese comics. Manga translation involves two important problems in machine translation: context-aware and multimodal translation. Since text and images are mixed up in an unstructured fashion in Manga, obtaining context from the image is essential for manga translation. However, it is still an open problem how to extract context from image and integrate into MT models. In addition, corpus and benchmarks to train and evaluate such model is currently unavailable. In this paper, we make the following four contributions that establishes the foundation of manga translation research. First, we propose multimodal context-aware translation framework. We are the first to incorporate context information obtained from manga image. It enables us to translate texts in speech bubbles that cannot be translated without using context information (e.g., texts in other speech bubbles, gender of speakers, etc.). Second, for training the model, we propose the approach to automatic corpus construction from pairs of original manga and their translations, by which large parallel corpus can be constructed without any manual labeling. Third, we created a new benchmark to evaluate manga translation. Finally, on top of our proposed methods, we devised a first comprehensive system for fully automated manga translation.