mlsanityData to 7 Oct 2026

CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs

arXiv 1911.06154 ↗First posted 10 Nov 2019cs.CL cs.LG stat.MLcited by 136

Not on the map. Why?

Abstract (arXiv)

Cross-lingual document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. In this paper, we exploit the signals embedded in URLs to label web documents at scale with an average precision of 94.5% across different language pairs. We mine sixty-eight snapshots of the Common Crawl corpus and identify web document pairs that are translations of each other. We release a new web dataset consisting of over 392 million URL pairs from Common Crawl covering documents in 8144 language pairs of which 137 pairs include English. In addition to curating this massive dataset, we introduce baseline methods that leverage cross-lingual representations to identify aligned documents based on their textual content. Finally, we demonstrate the value of this parallel documents dataset through a downstream task of mining parallel sentences and measuring the quality of machine translations from models trained on this mined data. Our objective in releasing this dataset is to foster new research in cross-lingual NLP across a variety of low, medium, and high-resource languages.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 31; 2023: 16; 2024: 29; 2025: 17; 2026: 9.

136 papers.

Sort:

By area:

  1. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  2. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  3. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  4. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  5. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 176
  6. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  7. 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
  8. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 133
  9. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  10. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  11. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  12. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
38 more
  1. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  2. 2024Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Modelscited by 69
  3. 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
  4. 2023Tokenization and the Noiseless Channelcited by 58
  5. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  6. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55
  7. 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 55
  8. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 55
  9. 2022A Survey on Cross-Lingual Summarizationcited by 38
  10. 2022AfroLID: A Neural Language Identification Tool for African Languagescited by 38
  11. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  12. 2023A Formal Perspective on Byte-Pair Encodingcited by 35
  13. 2021Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Taskcited by 34
  14. 2022Frustratingly Easy Label Projection for Cross-lingual Transfercited by 33
  15. 2023The ROOTS Search Tool: Data Transparency for LLMscited by 30
  16. 2022Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarizationcited by 29
  17. 2024Code-Switching Curriculum Learning for Multilingual Transfer in LLMscited by 26
  18. 2022Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?cited by 24
  19. 2024Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Interventioncited by 24
  20. 2021PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretrainingcited by 21
  21. 2022Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settingscited by 21
  22. 2022TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languagescited by 21
  23. 2022GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminatorcited by 20
  24. 2025Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMscited by 20
  25. 2021Machine Translation into Low-resource Language Varietiescited by 19
  26. 2021Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoningcited by 19
  27. 2021Improving Large-scale Language Models and Resources for Filipinocited by 19
  28. 2024Sailor: Open Language Models for South-East Asiacited by 19
  29. 2024Fairness and Bias in Multimodal AI: A Surveycited by 19
  30. 2021Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Datacited by 17
  31. 2022CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translationcited by 17
  32. 2021A Survey on Low-Resource Neural Machine Translationcited by 16
  33. 2022SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languagescited by 16
  34. 2022Beyond English-Centric Bitexts for Better Multilingual Language Representation Learningcited by 16
  35. 2021Quality Estimation without Human-labeled Datacited by 15
  36. 2021XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignmentcited by 15
  37. 2023Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport Distillationcited by 15
  38. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15

50 most cited of 136 All 136 (JSON)