mlsanityData to 7 Oct 2026

In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine Translation Using Large Language Models

arXiv 2305.03573 ↗First posted 5 May 2023cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

The phenomena of in-context learning has typically been thought of as "learning from examples". In this work which focuses on Machine Translation, we present a perspective of in-context learning as the desired generation task maintaining coherency with its context, i.e., the prompt examples. We first investigate randomly sampled prompts across 4 domains, and find that translation performance improves when shown in-domain prompts. Next, we investigate coherency for the in-domain setting, which uses prompt examples from a moving window. We study this with respect to other factors that have previously been identified in the literature such as length, surface similarity and sentence embedding similarity. Our results across 3 models (GPTNeo2.7B, Bloom3B, XGLM2.9B), and three translation directions (textttenrightarrowtextttpt, de, fr) suggest that the long-term coherency of the prompts and the test sentence is a good indicator of downstream translation performance. In doing so, we demonstrate the efficacy of In-context Machine Translation for on-the-fly adaptation.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

22 of 40 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
10 more
  1. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  3. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  4. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  5. 2022In-context Examples Selection for Machine Translationcited by 172
  6. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  7. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  8. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  9. 2018MTNT: A Testbed for Machine Translation of Noisy Textcited by 54
  10. 2021Can Transformer Models Measure Coherence In Text? Re-Thinking the Shuffle Testcited by 17

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 6; 2025: 2.

10 papers.

  1. 2023Challenges and Applications of Large Language Modelscited by 251
  2. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  3. 2024Data-adaptive Differentially Private Prompt Synthesis for In-Context Learningcited by 10
  4. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translationcited by 8
  5. 2023Language Models as Knowledge Bases for Visual Word Sense Disambiguationcited by 3
  6. 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
  7. 2025TICL: Text-Embedding KNN For Speech In-Context Learning Unlocks Speech Recognition Abilities of Large Multimodal Modelscited by 3
  8. 2024Where does In-context Translation Happen in Large Language Modelscited by 1
  9. 2024Optimizing example selection for retrieval-augmented machine translation with translation memoriescited by 0
  10. 2024Guiding In-Context Learning of LLMs through Quality Estimation for Machine Translationcited by 0