mlsanityData to 7 Oct 2026

Prompting Large Language Model for Machine Translation: A Case Study

arXiv 2301.07069 ↗First posted 17 Jan 2023cs.CL cs.LGcited by 247

Not on the map. Why?

Abstract (arXiv)

Research on prompting has shown excellent performance with little or even no supervised training across many tasks. However, prompting for machine translation is still under-explored in the literature. We fill this gap by offering a systematic study on prompting strategies for translation, examining various factors for prompt template and demonstration example selection. We further explore the use of monolingual data and the feasibility of cross-lingual, cross-domain, and sentence-to-document transfer learning in prompting. Extensive experiments with GLM-130B (Zeng et al., 2022) as the testbed show that 1) the number and the quality of prompt examples matter, where using suboptimal examples degenerates translation; 2) several features of prompt examples, such as semantic similarity, show significant Spearman correlation with their prompting performance; yet, none of the correlations are strong enough; 3) using pseudo parallel prompt examples constructed from monolingual data via zero-shot prompting could improve translation; and 4) improved performance is achievable by transferring knowledge from prompt examples selected in other settings. We finally provide an analysis on the model outputs and discuss several problems that prompting still suffers from.

Key facts. Cited by 247 papers in the map (data through 7 Oct 2026). Builds on 43 of its 66 references that are in the map. First posted on arXiv 17 Jan 2023.

Builds on

43 of 66 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  11. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
31 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  4. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  9. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  10. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  11. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  12. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  13. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  14. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  15. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  16. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  17. 2022Active Example Selection for In-Context Learningcited by 195
  18. 2022In-context Examples Selection for Machine Translationcited by 172
  19. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  20. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  21. 2019Tagged Back-Translationcited by 76
  22. 2018Guiding Neural Machine Translation with Retrieved Translation Piecescited by 63
  23. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  24. 2022Bidirectional Language Models Are Also Few-shot Learnerscited by 48
  25. 2021MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translatorscited by 35
  26. 2020Rethinking Document-level Neural Machine Translationcited by 32
  27. 2022Using natural language prompts for machine translationcited by 27
  28. 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24
  29. 2021The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21cited by 14
  30. 2022Extrinsic Evaluation of Machine Translation Metricscited by 14
  31. 2021WeChat Neural Machine Translation Systems for WMT21cited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 53; 2024: 112; 2025: 60; 2026: 22.

247 papers.

Sort:

By area:

  1. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  2. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  3. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  4. 2024Tool Learning with Large Language Models: A Surveycited by 221
  5. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  6. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
  7. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  8. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  9. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  10. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  11. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 97
  12. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 88
38 more
  1. 2023Do Multilingual Language Models Think Better in English?cited by 87
  2. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  3. 2024Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arenacited by 76
  4. 2024MarsCode Agent: AI-native Automated Bug Fixingcited by 68
  5. 2023Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memorycited by 64
  6. 2023Adaptive Machine Translation with Large Language Modelscited by 58
  7. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 58
  8. 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 57
  9. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 55
  10. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  11. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  12. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  13. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
  14. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  15. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  16. 2024TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedbackcited by 43
  17. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  18. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  19. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  20. 2023Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Coursescited by 38
  21. 2023SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learningcited by 38
  22. 2024TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinementcited by 37
  23. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  24. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34
  25. 2023Evaluating the Reversal Curse in Model Editingcited by 34
  26. 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 33
  27. 2023How to Design Translation Prompts for ChatGPT: An Empirical Studycited by 31
  28. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 31
  29. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  30. 2024Stepwise Alignment for Constrained Language Model Policy Optimizationcited by 30
  31. 2024Teaching Large Language Models an Unseen Language on the Flycited by 29
  32. 2024G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translationcited by 29
  33. 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
  34. 2023Escaping the sentence-level paradigm in machine translationcited by 27
  35. 2023Iterative Translation Refinement with Large Language Modelscited by 27
  36. 2024TasTe: Teaching Large Language Models to Translate through Self-Reflectioncited by 27
  37. 2024DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memorycited by 27
  38. 2023CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Modelscited by 26

50 most cited of 247 All 247 (JSON)