# Using natural language prompts for machine translation

arXiv: https://arxiv.org/abs/2202.11822 · HTML: https://mlsanity.com/paper/2202.11822/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation](https://mlsanity.com/paper/2004.11867/) (2020)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://mlsanity.com/paper/1907.05019/) (2019)
- [Muppet: Massive Multi-task Representations with Pre-Finetuning](https://mlsanity.com/paper/2101.11038/) (2021)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://mlsanity.com/paper/1506.07285/) (2015)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://mlsanity.com/paper/1601.01073/) (2016)
- [Tagged Back-Translation](https://mlsanity.com/paper/1906.06442/) (2019)
- [The Missing Ingredient in Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/1903.07091/) (2019)
- [Language Tags Matter for Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/2106.07930/) (2021)
- [Multilingual Neural Machine Translation with Task-Specific Attention](https://mlsanity.com/paper/1806.03280/) (2018)
- [A Multilingual View of Unsupervised Machine Translation](https://mlsanity.com/paper/2002.02955/) (2020)
- [Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages](https://mlsanity.com/paper/2009.11201/) (2020)
- [Reference Language based Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2004.02127/) (2020)

## Built on by (most cited first, 27 of 27)

- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [In-context Examples Selection for Machine Translation](https://mlsanity.com/paper/2212.02437/) (2022)
- [What Makes Good Examples for Visual In-Context Learning?](https://mlsanity.com/paper/2301.13670/) (2023)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM](https://mlsanity.com/paper/2303.01911/) (2023)
- [Benchmarking Machine Translation with Cultural Awareness](https://mlsanity.com/paper/2305.14328/) (2023)
- [Exploring Effective Factors for Improving Visual In-Context Learning](https://mlsanity.com/paper/2304.04748/) (2023)
- [TasTe: Teaching Large Language Models to Translate through Self-Reflection](https://mlsanity.com/paper/2406.08434/) (2024)
- [DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory](https://mlsanity.com/paper/2410.08143/) (2024)
- [Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis](https://mlsanity.com/paper/2502.11544/) (2025)
- [Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models](https://mlsanity.com/paper/2410.05661/) (2024)
- [FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation](https://mlsanity.com/paper/2210.00193/) (2022)
- [Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction](https://mlsanity.com/paper/2301.10309/) (2023)
- [Chain-of-Dictionary Prompting Elicits Translation in Large Language Models](https://mlsanity.com/paper/2305.06575/) (2023)
- [DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules](https://mlsanity.com/paper/2305.13406/) (2023)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)
- [CODET: A Benchmark for Contrastive Dialectal Evaluation of Machine Translation](https://mlsanity.com/paper/2305.17267/) (2023)
- [Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Prompts](https://mlsanity.com/paper/2209.11409/) (2022)
- [LLM-based Translation Inference with Iterative Bilingual Understanding](https://mlsanity.com/paper/2410.12543/) (2024)
- [DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decoding](https://mlsanity.com/paper/2212.04205/) (2022)
- [Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models](https://mlsanity.com/paper/2411.01141/) (2024)
- [Steering Large Language Models for Machine Translation Personalization](https://mlsanity.com/paper/2505.16612/) (2025)
- [FAME-MT Dataset: Formality Awareness Made Easy for Machine Translation Purposes](https://mlsanity.com/paper/2405.11942/) (2024)
- [Changes from Classical Statistics to Modern Statistics and Data Science](https://mlsanity.com/paper/2211.03756/) (2022)
- [SLoW: Select Low-frequency Words! Automatic Dictionary Selection for Translation on Large Language Models](https://mlsanity.com/paper/2507.18902/) (2025)

## Abstract (arXiv)

We explore the use of natural language prompts for controlling various aspects of the outputs generated by machine translation models. We demonstrate that natural language prompts allow us to influence properties like formality or specific dialect of the output. We show that using language names to control the output language of multilingual translation models enables positive transfer for unseen language pairs. This unlocks the ability to translate into languages not seen during fine-tuning by using their English names. We investigate how scale, number of pre-training steps, number of languages in fine-tuning, and language similarity affect this phenomenon.