mlsanityData to 14 Oct 2026

ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation

arXiv 2606.05421 ↗First posted 3 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

When a text is translated, does the translation retain the complexity of the original? We introduce ComplexityMT, a new challenge for assessing how text complexity and machine translation interact with and influence each other, using the Common European Framework of Reference for Languages (CEFR) levels as the measure of text complexity. Across six languages, including Arabic, Dutch, English, French, Hindi, and Russian, we evaluate three open-weight models, one closed model, and a commercial machine translation system on two tasks: i) correlation of CEFR with translation difficulty, and ii) shifts in CEFR levels of the source texts. Our experiments show that higher CEFR levels make texts more difficult to translate and that machine translation lowers the target text's CEFR level for most languages at the document level. These findings provide new insights for researchers and practitioners working on multilingual pedagogical content generation and MT difficulty estimation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 54 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

20 of 54 references.

References · 20

  1. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  2. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  3. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  4. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  5. 2019Text Readability Assessment for Second Language Learnerscited by 54
  6. 2020Controllable Text Simplification with Explicit Paraphrasingcited by 53
  7. 2020Linguistic Features for Readability Assessmentcited by 35
  8. 2019Controlling Text Complexity in Neural Machine Translationcited by 32
  9. 2021Trends, Limitations and Open Challenges in Automatic Readability Assessment Researchcited by 28
  10. 2022CEFR-Based Sentence Difficulty Annotation and Assessmentcited by 28
  11. 2020It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Informationcited by 26
  12. 2018A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplificationcited by 24
8 more
  1. 2022CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formalitycited by 24
  2. 2023ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessmentcited by 19
  3. 2021An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languagescited by 14
  4. 2024Analysing Zero-Shot Readability-Controlled Sentence Simplificationcited by 12
  5. 2020Simplify-then-Translate: Automatic Preprocessing for Black-Box Machine Translationcited by 7
  6. 2025Estimating Machine Translation Difficultycited by 6
  7. 2025UniversalCEFR: Enabling Open Multilingual Research on Language Proficiency Assessmentcited by 5
  8. 2024Machine Translation Models are Zero-Shot Detectors of Translation Directioncited by 3

Built on by

Not cited yet.