# A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism

arXiv: https://arxiv.org/abs/2401.05749 · HTML: https://mlsanity.com/paper/2401.05749/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB](https://mlsanity.com/paper/1911.04944/) (2019)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://mlsanity.com/paper/2004.14564/) (2020)
- [What Language Model to Train if You Have One Million GPU Hours?](https://mlsanity.com/paper/2210.15424/) (2022)
- [Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://mlsanity.com/paper/1808.07048/) (2018)
- [Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus](https://mlsanity.com/paper/2010.14571/) (2020)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://mlsanity.com/paper/1808.10432/) (2018)
- [Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metric](https://mlsanity.com/paper/2209.13654/) (2022)
- [Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora](https://mlsanity.com/paper/1809.00197/) (2018)
- [Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity](https://mlsanity.com/paper/2008.04935/) (2020)
- [Low-Resource Corpus Filtering using Multilingual Sentence Embeddings](https://mlsanity.com/paper/1906.08885/) (2019)
- [There's no Data Like Better Data: Using QE Metrics for MT Data Filtering](https://mlsanity.com/paper/2311.05350/) (2023)
- [Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing](https://mlsanity.com/paper/2212.12137/) (2022)
- [Exploiting Sentence Order in Document Alignment](https://mlsanity.com/paper/2004.14523/) (2020)

## Built on by (most cited first, 30 of 35)

- [X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale](https://mlsanity.com/paper/2410.03115/) (2024)
- [Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains](https://mlsanity.com/paper/2402.18747/) (2024)
- [Regurgitative Training: The Value of Real Data in Training Large Language Models](https://mlsanity.com/paper/2407.12835/) (2024)
- [Scaling Low-Resource MT via Synthetic Data Generation with LLMs](https://mlsanity.com/paper/2505.14423/) (2025)
- [SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translation](https://mlsanity.com/paper/2505.16637/) (2025)
- [Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?](https://mlsanity.com/paper/2410.09644/) (2024)
- [Enhancing Translation Accuracy of Large Language Models through Continual Pre-Training on Parallel Data](https://mlsanity.com/paper/2407.03145/) (2024)
- [A Review of the Challenges with Massive Web-mined Corpora Used in Large Language Models Pre-Training](https://mlsanity.com/paper/2407.07630/) (2024)
- [From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora](https://mlsanity.com/paper/2505.14045/) (2025)
- [Linguistic Characteristics of AI-Generated Text: A Survey](https://mlsanity.com/paper/2510.05136/) (2025)
- [Sociotechnical Implications of Generative Artificial Intelligence for Information Access](https://mlsanity.com/paper/2405.11612/) (2024)
- [Recovering document annotations for sentence-level bitext](https://mlsanity.com/paper/2406.03869/) (2024)
- [The Impact of AI-Generated Text on the Internet](https://mlsanity.com/paper/2604.26965/) (2026)
- [Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysis](https://mlsanity.com/paper/2405.14277/) (2024)
- [Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimization](https://mlsanity.com/paper/2409.17673/) (2024)
- [The Art of Asking: Multilingual Prompt Optimization for Synthetic Data](https://mlsanity.com/paper/2510.19806/) (2025)
- [Large Language Models as 'Hidden Persuaders': Fake Product Reviews are Indistinguishable to Humans and Machines](https://mlsanity.com/paper/2506.13313/) (2025)
- [Large Language Models as Search Engines: Societal Challenges](https://mlsanity.com/paper/2512.08946/) (2025)
- [Translation in the Wild](https://mlsanity.com/paper/2505.23548/) (2025)
- [Generative AI in Sociological Research: State of the Discipline](https://mlsanity.com/paper/2511.16884/) (2025)
- [The Culture Funnel: You Can't Align What isn't in the Data](https://mlsanity.com/paper/2606.13808/) (2026)
- [Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection](https://mlsanity.com/paper/2501.14728/) (2025)
- [Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs](https://mlsanity.com/paper/2507.03933/) (2025)
- [How Persuasive Could LLMs Be? A First Study Combining Linguistic-Rhetorical Analysis and User Experiments](https://mlsanity.com/paper/2508.09614/) (2025)
- [AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains](https://mlsanity.com/paper/2601.15511/) (2026)
- [KletterMix: Climbing Toward High-Quality German Pretraining Data - The Full Report](https://mlsanity.com/paper/2606.03773/) (2026)
- [Leveraging Corpus Metadata to Detect Template-based Translation: An Exploratory Case Study of the Egyptian Arabic Wikipedia Edition](https://mlsanity.com/paper/2404.00565/) (2024)
- [Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model](https://mlsanity.com/paper/2511.02958/) (2025)
- [Dialects of Translationese Shape Language Model Learning](https://mlsanity.com/paper/2602.16469/) (2026)
- [A Dataset for Probing Translationese Preferences in English-to-Swedish Translation](https://mlsanity.com/paper/2603.08450/) (2026)

## Abstract (arXiv)

We show that content on the web is often translated into many languages, and the low quality of these multi-way translations indicates they were likely created using Machine Translation (MT). Multi-way parallel, machine generated content not only dominates the translations in lower resource languages; it also constitutes a large fraction of the total web content in those languages. We also find evidence of a selection bias in the type of content which is translated into many languages, consistent with low quality English content being translated en masse into many lower resource languages, via MT. Our work raises serious concerns about training models such as multilingual large language models on both monolingual and bilingual data scraped from the web.