mlsanityData to 14 Oct 2026

FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation

arXiv 2210.00193 ↗First posted 1 Oct 2022cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

We present FRMT, a new dataset and evaluation benchmark for Few-shot Region-aware Machine Translation, a type of style-targeted translation. The dataset consists of professional translations from English into two regional variants each of Portuguese and Mandarin Chinese. Source documents are selected to enable detailed analysis of phenomena of interest, including lexically distinct terms and distractor terms. We explore automatic evaluation metrics for FRMT and validate their correlation with expert human evaluation across both region-matched and mismatched rating scenarios. Finally, we present a number of baseline models for this task, and offer guidelines for how researchers can train, evaluate, and compare their own models. Our dataset and evaluation code are publicly available: https://bit.ly/frmt-task

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 1 Oct 2022.

Builds on

27 of 52 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  8. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  9. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  10. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 248
  11. 2019Massively Multilingual Neural Machine Translationcited by 246
  12. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
15 more
  1. 2017Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Modelscited by 57
  2. 2020TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restylingcited by 40
  3. 2018Multi-Task Neural Models for Translating Between Styles Within and Across Languagescited by 28
  4. 2018Extreme Adaptation for Personalized Neural Machine Translationcited by 27
  5. 2022Using natural language prompts for machine translationcited by 27
  6. 2021A Review of Human Evaluation for Style Transfercited by 19
  7. 2021Machine Translation into Low-resource Language Varietiescited by 19
  8. 2022Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learningcited by 19
  9. 2017Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss Germancited by 9
  10. 2018Neural Machine Translation into Language Varietiescited by 9
  11. 2020Text Style Transfer: A Review and Experimental Evaluationcited by 9
  12. 2021Towards Universality in Multilingual Text Rewritingcited by 7
  13. 2018A Neural Approach to Language Variety Translationcited by 6
  14. 2019Unsupervised Neural Dialect Translation with Commonality and Diversity Modelingcited by 5
  15. 2021Towards Personalised and Document-level Machine Translation of Dialoguecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 6; 2025: 4; 2026: 3.

18 papers.

Sort:

By area:

  1. 2023PaLM 2 Technical Reportcited by 1,162
  2. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  3. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  4. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  5. 2023Benchmarking Machine Translation with Cultural Awarenesscited by 53
  6. 2024Natural Language Processing for Dialects of a Language: A Surveycited by 40
  7. 2023Quantifying the Dialect Gap and its Correlates Across Languagescited by 33
  8. 2024The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilitiescited by 22
  9. 2022Dialect-robust Evaluation of Generated Textcited by 14
  10. 2024Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialoguescited by 11
  11. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translationcited by 8
  12. 2025An Interdisciplinary Approach to Human-Centered Machine Translationcited by 6
6 more
  1. 2025Towards Style Alignment in Cross-Cultural Translationcited by 4
  2. 2024Extracting Lexical Features from Dialects via Interpretable Dialect Classifierscited by 3
  3. 2026ALBA: A European Portuguese Benchmark for Evaluating Language and Linguistic Dimensions in Generative LLMscited by 3
  4. 2026P3B3: A Multi-Turn Conversational Benchmark for Measuring European and Brazilian Portuguese Variety Bias in LLMscited by 1
  5. 2025When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Contentcited by 0
  6. 2026On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lenscited by 0