# Simpson's Paradox and the Accuracy-Fluency Tradeoff in Translation

arXiv: https://arxiv.org/abs/2402.12690 · HTML: https://mlsanity.com/paper/2402.12690/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 57 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Domain Robustness in Neural Machine Translation](https://mlsanity.com/paper/1911.03109/) (2019)
- [Simple and Effective Noisy Channel Modeling for Neural Machine Translation](https://mlsanity.com/paper/1908.05731/) (2019)
- [Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT](https://mlsanity.com/paper/1802.06041/) (2018)
- [The Neural Noisy Channel](https://mlsanity.com/paper/1611.02554/) (2016)
- [Simpson's Bias in NLP Training](https://mlsanity.com/paper/2103.11795/) (2021)

## Built on by (most cited first, 8 of 8)

- [Speech is More Than Words: Do Speech-to-Text Translation Systems Leverage Prosody?](https://mlsanity.com/paper/2410.24019/) (2024)
- [Multi-Dimensional Machine Translation Evaluation: Model Evaluation and Resource for Korean](https://mlsanity.com/paper/2403.12666/) (2024)
- [You Cannot Feed Two Birds with One Score: the Accuracy-Naturalness Tradeoff in Translation](https://mlsanity.com/paper/2503.24013/) (2025)
- [MegaWika 2: A More Comprehensive Multilingual Collection of Articles and their Sources](https://mlsanity.com/paper/2508.03828/) (2025)
- [A Probability--Quality Trade-off in Aligned Language Models and its Relation to Sampling Adaptors](https://mlsanity.com/paper/2406.10203/) (2024)
- [Toward Machine Interpreting: Lessons from Human Interpreting Studies](https://mlsanity.com/paper/2508.07964/) (2025)
- [Dialects of Translationese Shape Language Model Learning](https://mlsanity.com/paper/2602.16469/) (2026)
- [Translation from the Information Bottleneck Perspective: an Efficiency Analysis of Spatial Prepositions in Bitexts](https://mlsanity.com/paper/2603.19924/) (2026)

## Abstract (arXiv)

A good translation should be faithful to the source and should respect the norms of the target language. We address a theoretical puzzle about the relationship between these objectives. On one hand, intuition and some prior work suggest that accuracy and fluency should trade off against each other, and that capturing every detail of the source can only be achieved at the cost of fluency. On the other hand, quality assessment researchers often suggest that accuracy and fluency are highly correlated and difficult for human raters to distinguish (Callison-Burch et al., 2007). We show that the tension between these views is an instance of Simpson's paradox, and that accuracy and fluency are positively correlated at the level of the corpus but trade off at the level of individual source segments. We further suggest that the relationship between accuracy and fluency is best evaluated at the segment (or sentence) level, and that the trade off between these dimensions has implications both for assessing translation quality and developing improved MT systems.