# Dialects of Translationese Shape Language Model Learning

arXiv: https://arxiv.org/abs/2602.16469 · HTML: https://mlsanity.com/paper/2602.16469/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Builds on (most cited first, 15 of 15)

- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Choosing Transfer Languages for Cross-Lingual Learning](https://mlsanity.com/paper/1905.12688/) (2019)
- [Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation](https://mlsanity.com/paper/2102.00287/) (2021)
- [Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca](https://mlsanity.com/paper/2309.08958/) (2023)
- [A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism](https://mlsanity.com/paper/2401.05749/) (2024)
- [A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank](https://mlsanity.com/paper/2205.04086/) (2022)
- [BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation](https://mlsanity.com/paper/2502.04314/) (2025)
- [Multilingual Language Model Pretraining using Machine-translated Data](https://mlsanity.com/paper/2502.13252/) (2025)
- [Confounding Factors in Relating Model Performance to Morphology](https://mlsanity.com/paper/2511.01380/) (2025)
- [To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation](https://mlsanity.com/paper/2401.01419/) (2024)
- [Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Matter](https://mlsanity.com/paper/2501.14491/) (2025)
- [Untangling the Influence of Typology, Data and Model Architecture on Ranking Transfer Languages for Cross-Lingual POS Tagging](https://mlsanity.com/paper/2503.19979/) (2025)
- [Simpson's Paradox and the Accuracy-Fluency Tradeoff in Translation](https://mlsanity.com/paper/2402.12690/) (2024)
- [On the Similarities Between Native, Non-native and Translated Texts](https://mlsanity.com/paper/1609.03204/) (2016)
- [Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish](https://mlsanity.com/paper/2602.03484/) (2026)

## Abstract (arXiv)

Machine-translated data is widely used in multilingual NLP, particularly where native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it reflects both traces of the source language and characteristic properties of translation itself. In this paper, we study how training on machine-translated data affects small English language models, focusing on how translationese from different source languages shapes linguistic acceptability judgments and language modeling for different domains. We train models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn. Our results show that the source language has a clear impact on model behavior: general perplexity is more driven by the lexical diversity of the translated corpus, but grammatical performance is strongly correlated to typological similarity to English if trained on enough data. Even translation quality is a strong predictor of language modeling performance.