# ChatGPT as a Translation Engine: A Case Study on Japanese-English

arXiv: https://arxiv.org/abs/2510.08042 · HTML: https://mlsanity.com/paper/2510.08042/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 12 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 5 of 5)

- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Towards Making the Most of ChatGPT for Machine Translation](https://mlsanity.com/paper/2303.13780/) (2023)
- [Optimizing Machine Translation through Prompt Engineering: An Investigation into ChatGPT's Customizability](https://mlsanity.com/paper/2308.01391/) (2023)
- [Difficulty-Aware Machine Translation Evaluation](https://mlsanity.com/paper/2107.14402/) (2021)

## Abstract (arXiv)

This study investigates ChatGPT for Japanese-English translation, exploring simple and enhanced prompts and comparing against commercially available translation engines. Performing both automatic and MQM-based human evaluations, we found that document-level translation outperforms sentence-level translation for ChatGPT. On the other hand, we were not able to determine if enhanced prompts performed better than simple prompts in our experiments. We also discovered that ChatGPT-3.5 was preferred by automatic evaluation, but a tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). Lastly, ChatGPT yields competitive results against two widely-known translation systems.