# CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality

arXiv: https://arxiv.org/abs/2205.04022 · HTML: https://mlsanity.com/paper/2205.04022/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Using the Output Embedding to Improve Language Models](https://mlsanity.com/paper/1608.05859/) (2016)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB](https://mlsanity.com/paper/1911.04944/) (2019)
- [Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset](https://mlsanity.com/paper/1909.05358/) (2019)
- [Achieving Human Parity on Automatic Chinese to English News Translation](https://mlsanity.com/paper/1803.05567/) (2018)
- [Getting Gender Right in Neural Machine Translation](https://mlsanity.com/paper/1909.05088/) (2019)
- [Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus](https://mlsanity.com/paper/2006.05754/) (2020)
- [MTNT: A Testbed for Machine Translation of Noisy Text](https://mlsanity.com/paper/1809.00388/) (2018)
- [Personalized Machine Translation: Preserving Original Author Traits](https://mlsanity.com/paper/1610.05461/) (2016)
- [Controlling Text Complexity in Neural Machine Translation](https://mlsanity.com/paper/1911.00835/) (2019)
- [Multi-Task Neural Models for Translating Between Styles Within and Across Languages](https://mlsanity.com/paper/1806.04357/) (2018)
- [Extreme Adaptation for Personalized Neural Machine Translation](https://mlsanity.com/paper/1805.01817/) (2018)
- [JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus](https://mlsanity.com/paper/1911.10668/) (2019)
- [Gender Aware Spoken Language Translation Applied to English-Arabic](https://mlsanity.com/paper/1802.09287/) (2018)
- [Controlling Neural Machine Translation Formality with Synthetic Supervision](https://mlsanity.com/paper/1911.08706/) (2019)
- [Machine Translation Verbosity Control for Automatic Dubbing](https://mlsanity.com/paper/2110.03847/) (2021)
- [The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020](https://mlsanity.com/paper/2008.04885/) (2020)
- [Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1](https://mlsanity.com/paper/2006.02964/) (2020)

## Built on by (most cited first, 24 of 24)

- [MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation](https://mlsanity.com/paper/2211.01355/) (2022)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Controlling Translation Formality Using Pre-trained Multilingual Language Models](https://mlsanity.com/paper/2205.06644/) (2022)
- [Identifying Context-Dependent Translations for Evaluation Set Production](https://mlsanity.com/paper/2311.02321/) (2023)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)
- [Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation](https://mlsanity.com/paper/2503.06594/) (2025)
- [Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation](https://mlsanity.com/paper/2410.13944/) (2024)
- [Controlling Formality in Low-Resource NMT with Domain Adaptation and Re-Ranking: SLT-CDT-UoS at IWSLT2022](https://mlsanity.com/paper/2205.05990/) (2022)
- [Towards Style Alignment in Cross-Cultural Translation](https://mlsanity.com/paper/2507.00216/) (2025)
- [Isolating Culture Neurons in Multilingual Large Language Models](https://mlsanity.com/paper/2508.02241/) (2025)
- [Sockeye 3: Fast Neural Machine Translation with PyTorch](https://mlsanity.com/paper/2207.05851/) (2022)
- [Steering Large Language Models for Machine Translation Personalization](https://mlsanity.com/paper/2505.16612/) (2025)
- [In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages](https://mlsanity.com/paper/2302.12299/) (2023)
- [MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by Leveraging Unstructured Context in Neural Machine Translation](https://mlsanity.com/paper/2305.15904/) (2023)
- [How Transferable are Attribute Controllers on Pretrained Multilingual Translation Models?](https://mlsanity.com/paper/2309.08565/) (2023)
- [On Instruction-Finetuning Neural Machine Translation Models](https://mlsanity.com/paper/2410.05553/) (2024)
- [A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective](https://mlsanity.com/paper/2402.12627/) (2024)
- [FAME-MT Dataset: Formality Awareness Made Easy for Machine Translation Purposes](https://mlsanity.com/paper/2405.11942/) (2024)
- [Beyond "To whom it may concern": Tailoring Machine Translation to Audience and Intent](https://mlsanity.com/paper/2606.03259/) (2026)
- [Data-Driven Approach for Formality-Sensitive Machine Translation: Language-Specific Handling and Synthetic Data Generation](https://mlsanity.com/paper/2306.14514/) (2023)
- [ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation](https://mlsanity.com/paper/2606.05421/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing](https://mlsanity.com/paper/2609.25298/) (2026)
- [Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following](https://mlsanity.com/paper/2609.28395/) (2026)

## Abstract (arXiv)

The machine translation (MT) task is typically formulated as that of returning a single translation for an input segment. However, in many cases, multiple different translations are valid and the appropriate translation may depend on the intended target audience, characteristics of the speaker, or even the relationship between speakers. Specific problems arise when dealing with honorifics, particularly translating from English into languages with formality markers. For example, the sentence "Are you sure?" can be translated in German as "Sind Sie sich sicher?" (formal register) or "Bist du dir sicher?" (informal). Using wrong or inconsistent tone may be perceived as inappropriate or jarring for users of certain cultures and demographics. This work addresses the problem of learning to control target language attributes, in this case formality, from a small amount of labeled contrastive data. We introduce an annotated dataset (CoCoA-MT) and an associated evaluation metric for training and evaluating formality-controlled MT models for six diverse target languages. We show that we can train formality-controlled models by fine-tuning on labeled contrastive data, achieving high accuracy (82% in-domain and 73% out-of-domain) while maintaining overall quality.