# How to Design Translation Prompts for ChatGPT: An Empirical Study

arXiv: https://arxiv.org/abs/2304.02182 · HTML: https://mlsanity.com/paper/2304.02182/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Towards Making the Most of ChatGPT for Machine Translation](https://mlsanity.com/paper/2303.13780/) (2023)
- [Analyzing Uncertainty in Neural Machine Translation](https://mlsanity.com/paper/1803.00047/) (2018)
- [Neural Machine Translation: A Review and Survey](https://mlsanity.com/paper/1912.02047/) (2019)
- [Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text](https://mlsanity.com/paper/2107.01294/) (2021)
- [A Survey of Deep Learning Techniques for Neural Machine Translation](https://mlsanity.com/paper/2002.07526/) (2020)
- [Incorporating BERT into Parallel Sequence Decoding with Adapters](https://mlsanity.com/paper/2010.06138/) (2020)
- [GPT3-to-plan: Extracting plans from text using GPT-3](https://mlsanity.com/paper/2106.07131/) (2021)
- [Revisiting Self-Training for Few-Shot Learning of Language Model](https://mlsanity.com/paper/2110.01256/) (2021)
- [Controlling Neural Machine Translation Formality with Synthetic Supervision](https://mlsanity.com/paper/1911.08706/) (2019)
- [Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning](https://mlsanity.com/paper/1708.00993/) (2017)
- [Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring](https://mlsanity.com/paper/1908.05925/) (2019)
- [Learning from Chunk-based Feedback in Neural Machine Translation](https://mlsanity.com/paper/1806.07169/) (2018)
- [Improving English to Sinhala Neural Machine Translation using Part-of-Speech Tag](https://mlsanity.com/paper/2202.08882/) (2022)

## Built on by (most cited first, 30 of 31)

- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP](https://mlsanity.com/paper/2305.14976/) (2023)
- [When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks](https://mlsanity.com/paper/2311.08993/) (2023)
- [An In-depth Look at Gemini's Language Abilities](https://mlsanity.com/paper/2312.11444/) (2023)
- [Large Language Models are Complex Table Parsers](https://mlsanity.com/paper/2312.11521/) (2023)
- [ChatGPT MT: Competitive for High- (but not Low-) Resource Languages](https://mlsanity.com/paper/2309.07423/) (2023)
- [TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties](https://mlsanity.com/paper/2308.03051/) (2023)
- [An Empirical Study of Translation Hypothesis Ensembling with Large Language Models](https://mlsanity.com/paper/2310.11430/) (2023)
- [What is the Best Way for ChatGPT to Translate Poetry?](https://mlsanity.com/paper/2406.03450/) (2024)
- [Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions](https://mlsanity.com/paper/2402.18025/) (2024)
- [Prompting ChatGPT for Translation: A Comparative Analysis of Translation Brief and Persona Prompts](https://mlsanity.com/paper/2403.00127/) (2024)
- [Optimizing Machine Translation through Prompt Engineering: An Investigation into ChatGPT's Customizability](https://mlsanity.com/paper/2308.01391/) (2023)
- [Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry](https://mlsanity.com/paper/2509.01620/) (2025)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [LLM-based Translation Inference with Iterative Bilingual Understanding](https://mlsanity.com/paper/2410.12543/) (2024)
- [Gender Bias in Machine Translation and The Era of Large Language Models](https://mlsanity.com/paper/2401.10016/) (2024)
- [Are AI agents the new machine translation frontier? Challenges and opportunities of single- and multi-agent systems for multilingual digital communication](https://mlsanity.com/paper/2504.12891/) (2025)
- [Can ChatGPT Really Understand Modern Chinese Poetry?](https://mlsanity.com/paper/2603.20851/) (2026)
- [Aligning Translation-Specific Understanding to General Understanding in Large Language Models](https://mlsanity.com/paper/2401.05072/) (2024)
- [ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning](https://mlsanity.com/paper/2507.07306/) (2025)
- [MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language](https://mlsanity.com/paper/2508.00673/) (2025)
- [Could We Have Had Better Multilingual LLMs If English Was Not the Central Language?](https://mlsanity.com/paper/2402.13917/) (2024)
- [Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMs](https://mlsanity.com/paper/2505.22293/) (2025)
- [Creativity Bias: How Machine Evaluation Struggles with Creativity in Literary Translations](https://mlsanity.com/paper/2605.13596/) (2026)
- [Missing the human touch? A computational stylometry analysis of GPT-4 translations of online Chinese literature](https://mlsanity.com/paper/2506.13013/) (2025)
- [Designing LLMs for cultural sensitivity: Evidence from English-Japanese translation](https://mlsanity.com/paper/2509.11921/) (2025)
- [Multilingual LLM Prompting Strategies for Medical English-Vietnamese Machine Translation](https://mlsanity.com/paper/2509.15640/) (2025)
- [When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Content](https://mlsanity.com/paper/2512.17738/) (2025)
- [AI-assisted cultural heritage dissemination: Comparing NMT and glossary-augmented LLM translation in rock art documents](https://mlsanity.com/paper/2605.14679/) (2026)

## Abstract (arXiv)

The recently released ChatGPT has demonstrated surprising abilities in natural language understanding and natural language generation. Machine translation relies heavily on the abilities of language understanding and generation. Thus, in this paper, we explore how to assist machine translation with ChatGPT. We adopt several translation prompts on a wide range of translations. Our experimental results show that ChatGPT with designed translation prompts can achieve comparable or better performance over commercial translation systems for high-resource language translations. We further evaluate the translation quality using multiple references, and ChatGPT achieves superior performance compared to commercial systems. We also conduct experiments on domain-specific translations, the final results show that ChatGPT is able to comprehend the provided domain keyword and adjust accordingly to output proper translations. At last, we perform few-shot prompts that show consistent improvement across different base prompts. Our work provides empirical evidence that ChatGPT still has great potential in translations.