mlsanityData to 7 Oct 2026

A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism

arXiv 2401.05749 ↗First posted 11 Jan 2024cs.CL cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

We show that content on the web is often translated into many languages, and the low quality of these multi-way translations indicates they were likely created using Machine Translation (MT). Multi-way parallel, machine generated content not only dominates the translations in lower resource languages; it also constitutes a large fraction of the total web content in those languages. We also find evidence of a selection bias in the type of content which is translated into many languages, consistent with low quality English content being translated en masse into many lower resource languages, via MT. Our work raises serious concerns about training models such as multilingual large language models on both monolingual and bilingual data scraped from the web.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 27 of its 56 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

27 of 56 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  9. 2023The Falcon Series of Open Language Modelscited by 526
  10. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  11. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  12. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
15 more
  1. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  2. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  3. 2020Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasingcited by 114
  4. 2022What Language Model to Train if You Have One Million GPU Hours?cited by 92
  5. 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
  6. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  7. 2020Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpuscited by 76
  8. 2018Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translationcited by 58
  9. 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
  10. 2018Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corporacited by 40
  11. 2020Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversitycited by 35
  12. 2019Low-Resource Corpus Filtering using Multilingual Sentence Embeddingscited by 25
  13. 2023There's no Data Like Better Data: Using QE Metrics for MT Data Filteringcited by 19
  14. 2022Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbingcited by 17
  15. 2020Exploiting Sentence Order in Document Alignmentcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 13; 2026: 9.

33 papers.

Sort:

By area:

  1. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  2. 2024Fine-Tuned Machine Translation Metrics Struggle in Unseen Domainscited by 21
  3. 2024Regurgitative Training: The Value of Real Data in Training Large Language Modelscited by 15
  4. 2025Scaling Low-Resource MT via Synthetic Data Generation with LLMscited by 14
  5. 2025SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translationcited by 13
  6. 2024Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?cited by 9
  7. 2024Enhancing Translation Accuracy of Large Language Models through Continual Pre-Training on Parallel Datacited by 7
  8. 2024A Review of the Challenges with Massive Web-mined Corpora Used in Large Language Models Pre-Trainingcited by 7
  9. 2025Linguistic Characteristics of AI-Generated Text: A Surveycited by 7
  10. 2024Sociotechnical Implications of Generative Artificial Intelligence for Information Accesscited by 6
  11. 2024Recovering document annotations for sentence-level bitextcited by 6
  12. 2025From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corporacited by 6
21 more
  1. 2026The Impact of AI-Generated Text on the Internetcited by 6
  2. 2024Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysiscited by 5
  3. 2024Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimizationcited by 5
  4. 2025The Art of Asking: Multilingual Prompt Optimization for Synthetic Datacited by 5
  5. 2025Large Language Models as Search Engines: Societal Challengescited by 3
  6. 2025Translation in the Wildcited by 2
  7. 2025Generative AI in Sociological Research: State of the Disciplinecited by 2
  8. 2026The Culture Funnel: You Can't Align What isn't in the Datacited by 2
  9. 2025Large Language Models as 'Hidden Persuaders': Fake Product Reviews are Indistinguishable to Humans and Machinescited by 1
  10. 2025Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMscited by 1
  11. 2026AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domainscited by 1
  12. 2024Leveraging Corpus Metadata to Detect Template-based Translation: An Exploratory Case Study of the Egyptian Arabic Wikipedia Editioncited by 0
  13. 2025Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detectioncited by 0
  14. 2025How Persuasive Could LLMs Be? A First Study Combining Linguistic-Rhetorical Analysis and User Experimentscited by 0
  15. 2025Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Modelcited by 0
  16. 2026Dialects of Translationese Shape Language Model Learningcited by 0
  17. 2026A Dataset for Probing Translationese Preferences in English-to-Swedish Translationcited by 0
  18. 2026KletterMix: Climbing Toward High-Quality German Pretraining Data - The Full Reportcited by 0
  19. 2026A Human-Centred Approach to Benchmarking LLMs for Parenting Advicecited by 0
  20. 2026The evolution of sex for artificial intelligence: a population-genetic framework for multigenerational model populationscited by 0
  21. 2026You Can Tell Who's Asking: What the Web's Questions Are Made Of, and Where They Come Fromcited by 0