mlsanityData to 7 Oct 2026

The Translation Tax Is Not a Scalar: A Counterfactual Audit of English-Source Cue Inheritance in Chinese Multilingual Benchmarks

arXiv 2605.07093 ↗First posted 8 May 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The Translation Tax is often treated as a scalar: translated benchmarks are assumed to inflate scores by preserving English-source cues. We audit this claim in an English-to-Chinese setting. Three proxy estimators disagree: back-translation gaps are small and parser-fragile; cue-score calibration does not predict item-level gains; and a six-model native-control comparison shows model-family rather than uniform benchmark effects. We add a same-item LLM-naturalization stress test that holds answer, options, and content fixed while rewriting Chinese surface form. After correcting a prompt-construction bug, this contrast no longer supports a model-family interaction, but it preserves a residue dose-response: high-residue items benefit while low-residue items do not. The result is not a single Translation Tax, but a set of estimator- and item-dependent validity risks. We release per-cell evidence, the naturalization protocol, human QC, and a reporting checklist for translated multilingual benchmark papers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 11 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

10 of 11 references.

References · 10

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  3. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  4. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  5. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  6. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  7. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
  8. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 95
  9. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  10. 2025The Bitter Lesson Learned from 2,000+ Multilingual Benchmarkscited by 31

Built on by

Not cited yet.