mlsanityData to 7 Oct 2026

In-context Examples Selection for Machine Translation

arXiv 2212.02437 ↗First posted 5 Dec 2022cs.CLcited by 172

Not on the map. Why?

Abstract (arXiv)

Large-scale generative models show an impressive ability to perform a wide range of Natural Language Processing (NLP) tasks using in-context learning, where a few examples are used to describe a task to the model. For Machine Translation (MT), these examples are typically randomly sampled from the development dataset with a similar distribution as the evaluation set. However, it is unclear how the choice of these in-context examples and their ordering impacts the output translation quality. In this work, we aim to understand the properties of good in-context examples for MT in both in-domain and out-of-domain settings. We show that the translation quality and the domain of the in-context examples matter and that 1-shot noisy unrelated example can have a catastrophic impact on output quality. While concatenating multiple random examples reduces the effect of noise, a single good prompt optimized to maximize translation quality on the development dataset can elicit learned information from the pre-trained language model. Adding similar examples based on an n-gram overlap with the test source significantly and consistently improves the translation quality of the outputs, outperforming a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets.

Key facts. Cited by 172 papers in the map (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 5 Dec 2022.

Builds on

33 of 49 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  11. 2019How Can We Know What Language Models Know?cited by 953
  12. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
21 more
  1. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  3. 2017Six Challenges for Neural Machine Translationcited by 334
  4. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  5. 2020Nearest Neighbor Machine Translationcited by 209
  6. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  7. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  8. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  9. 2020Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasingcited by 114
  10. 2022Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Datacited by 61
  11. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  12. 2016Fast Domain Adaptation for Neural Machine Translationcited by 28
  13. 2022Using natural language prompts for machine translationcited by 27
  14. 2021Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translationcited by 20
  15. 2020Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rankcited by 18
  16. 2022Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsingcited by 15
  17. 2022Towards Robust k-Nearest-Neighbor Machine Translationcited by 15
  18. 2022PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learningcited by 14
  19. 2018Identifying Semantic Divergences in Parallel Text without Annotationscited by 9
  20. 2021Assessing Reference-Free Peer Evaluation for Machine Translationcited by 8
  21. 2020CoRT: Complementary Rankings from Transformerscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 54; 2024: 73; 2025: 29; 2026: 15.

172 papers.

Sort:

By area:

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  2. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  3. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  4. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  5. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  6. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  7. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  8. 2023What Makes Good Examples for Visual In-Context Learning?cited by 150
  9. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  10. 2023Compositional Exemplars for In-context Learningcited by 143
  11. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  12. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
38 more
  1. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  2. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
  3. 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
  4. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  5. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  6. 2023Adaptive Machine Translation with Large Language Modelscited by 58
  7. 2023Coverage-based Example Selection for In-Context Learningcited by 58
  8. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 55
  9. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  10. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  11. 2023TIM: Teaching Large Language Models to Translate with Comparisoncited by 50
  12. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  13. 2023In-Context Learning Learns Label Relationships but Is Not Conventional Learningcited by 48
  14. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  15. 2024Why Larger Language Models Do In-context Learning Differently?cited by 44
  16. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  17. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  18. 2023Using In-Context Learning to Improve Dialogue Safetycited by 41
  19. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  20. 2023MoT: Memory-of-Thought Enables ChatGPT to Self-Improvecited by 37
  21. 2023Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Provingcited by 37
  22. 2024TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinementcited by 37
  23. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34
  24. 2023BLESS: Benchmarking Large Language Models on Sentence Simplificationcited by 34
  25. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
  26. 2024Teaching Large Language Models an Unseen Language on the Flycited by 29
  27. 2024G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translationcited by 29
  28. 2023Iterative Translation Refinement with Large Language Modelscited by 27
  29. 2024TasTe: Teaching Large Language Models to Translate through Self-Reflectioncited by 27
  30. 2024Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learningcited by 23
  31. 2024The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilitiescited by 22
  32. 2023Augmenting Large Language Model Translators via Translation Memoriescited by 21
  33. 2023Fine-tuning Large Language Models for Adaptive Machine Translationcited by 21
  34. 2024In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translationcited by 21
  35. 2024Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descentcited by 21
  36. 2023ScatterShot: Interactive In-context Example Curation for Text Transformationcited by 20
  37. 2023Supervised Knowledge Makes Large Language Models Better In-context Learnerscited by 20
  38. 2025DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answeringcited by 20

50 most cited of 172 All 172 (JSON)