mlsanityData to 14 Oct 2026

CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation

arXiv 2305.14105 ↗First posted 23 May 2023cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Large language models have demonstrated the capability to perform on machine translation when the input is prompted with a few examples (in-context learning). Translation quality depends on various features of the selected examples, such as their quality and relevance, but previous work has predominantly focused on individual features in isolation. In this paper, we propose a general framework for combining different features influencing example selection. We learn a regression model, CTQ Scorer (Contextual Translation Quality), that selects examples based on multiple features in order to maximize the translation quality. On multiple language pairs and language models, we show that CTQ Scorer helps significantly outperform random selection as well as strong single-factor baselines reported in the literature. We also see an improvement of over 2.5 COMET points on average with respect to a strong BM25 retrieval-based baseline.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 22 of its 37 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

22 of 37 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  11. 2019How Can We Know What Language Models Know?cited by 959
  12. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
10 more
  1. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  3. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  4. 2022News Summarization and Evaluation in the Era of GPT-3cited by 327
  5. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  6. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  7. 2022In-context Examples Selection for Machine Translationcited by 172
  8. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  9. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  10. 2022Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsingcited by 15

Built on by

Cited by year

Citing papers per year: 2024: 10; 2025: 2.

12 papers.

  1. 2024Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Languagecited by 12
  2. 2024Monotonic Paraphrasing Improves Generalization of Language Model Promptingcited by 10
  3. 2024An Empirical Study of In-context Learning in LLMs for Machine Translationcited by 9
  4. 2024FamiCom: Further Demystifying Prompts for Language Models with Task-Agnostic Performance Estimationcited by 8
  5. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translationcited by 8
  6. 2024Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparisoncited by 7
  7. 2024Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptionscited by 2
  8. 2025Exploring In-context Example Generation for Machine Translationcited by 2
  9. 2024SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translationcited by 1
  10. 2024Effective Self-Mining of In-Context Examples for Unsupervised Machine Translation with LLMscited by 1
  11. 2024Going Beyond Word Matching: Syntax Improves In-context Example Selection for Machine Translationcited by 0
  12. 2024Guiding In-Context Learning of LLMs through Quality Estimation for Machine Translationcited by 0