mlsanityData to 14 Oct 2026

Neural Machine Translation Models Can Learn to be Few-shot Learners

arXiv 2309.08590 ↗First posted 15 Sep 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

The emergent ability of Large Language Models to use a small number of examples to learn to perform in novel domains and tasks, also called in-context learning (ICL). In this work, we show that a much smaller model can be trained to perform ICL by fine-tuning towards a specialized training objective, exemplified on the task of domain adaptation for neural machine translation. With this capacity for ICL, the model can take advantage of relevant few-shot examples to adapt its output towards the domain. We compare the quality of this domain adaptation to traditional supervised techniques and ICL with a 40B-parameter Large Language Model. Our approach allows efficient batch inference on a mix of domains and outperforms state-of-the-art baselines in terms of both translation quality and immediate adaptation rate, i.e. the ability to reproduce a specific term after being shown a single example.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 15 Sep 2023.

Builds on

16 of 29 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  6. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  8. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  9. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
  10. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  11. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  12. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 145
4 more
  1. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  2. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 59
  3. 2023Adaptive Machine Translation with Large Language Modelscited by 58
  4. 2022Automatic Correction of Human Translationscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 6.

8 papers.

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  3. 2023Fine-tuning Large Language Models for Adaptive Machine Translationcited by 21
  4. 2023Assessing Translation capabilities of Large Language Models involving English and Indian Languagescited by 8
  5. 2024Context-Aware or Context-Insensitive? Assessing LLMs' Performance in Document-Level Translationcited by 6
  6. 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
  7. 2024Thinker-DDM: Modeling Deliberation for Machine Translation with a Drift-Diffusion Processcited by 3
  8. 2024A Context-aware Framework for Translation-mediated Conversationscited by 2