mlsanityData to 14 Oct 2026

Majority Voting with Bidirectional Pre-translation For Bitext Retrieval

arXiv 2103.06369 ↗First posted 10 Mar 2021cs.CL cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Obtaining high-quality parallel corpora is of paramount importance for training NMT systems. However, as many language pairs lack adequate gold-standard training data, a popular approach has been to mine so-called "pseudo-parallel" sentences from paired documents in two languages. In this paper, we outline some problems with current methods, propose computationally economical solutions to those problems, and demonstrate success with novel methods on the Tatoeba similarity search benchmark and on a downstream task, namely NMT. We uncover the effect of resource-related factors (i.e. how much monolingual/bilingual data is available for a given language) on the optimal choice of bitext mining approach, and echo problems with the oft-used BUCC dataset that have been observed by others. We make the code and data used for our experiments publicly available.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 56 of its 84 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

56 of 84 references.

References · 56

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2017Billion-scale similarity search with GPUscited by 2,497
  8. 2018Deep contextualized word representationscited by 2,290
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  10. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  11. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  12. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
38 more
  1. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  2. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  3. 2019How multilingual is Multilingual BERT?cited by 645
  4. 2019Green AIcited by 627
  5. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  6. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  7. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  8. 2017Word Translation Without Parallel Datacited by 487
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  11. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  12. 2015Skip-Thought Vectorscited by 322
  13. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  14. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
  15. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  16. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  17. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  18. 2017Unsupervised Neural Machine Translationcited by 149
  19. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 146
  20. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  21. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 128
  22. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
  23. 2020Multilingual Alignment of Contextual Word Representationscited by 114
  24. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  25. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 70
  26. 2014Improving zero-shot learning by mitigating the hubness problemcited by 54
  27. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  28. 2017Learning Joint Multilingual Sentence Representations with Neural Machine Translationcited by 52
  29. 2018Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learningcited by 52
  30. 2018Unsupervised Statistical Machine Translationcited by 51
  31. 2020When Does Unsupervised Machine Translation Work?cited by 38
  32. 2019An Effective Approach to Unsupervised Machine Translationcited by 37
  33. 2020Are All Good Word Vector Spaces Isomorphic?cited by 37
  34. 2020A Call for More Rigor in Unsupervised Cross-lingual Learningcited by 37
  35. 2020When and Why is Unsupervised Neural Machine Translation Useless?cited by 34
  36. 2018Filtering and Mining Parallel Data in a Joint Multilingual Spacecited by 32
  37. 2016Massively Multilingual Word Embeddingscited by 26
  38. 2019Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Frameworkcited by 26

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 1.

5 papers.

  1. 2021A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Spacecited by 8
  2. 2022Better Quality Estimation for Low Resource Corpus Miningcited by 8
  3. 2021"Wikily" Supervised Neural Translation Tailored to Cross-Lingual Taskscited by 5
  4. 2022AugCSE: Contrastive Sentence Embedding with Diverse Augmentationscited by 5
  5. 2023Replicable Benchmarking of Neural Machine Translation (NMT) on Low-Resource Local Languages in Indonesiacited by 0