mlsanityData to 14 Oct 2026

Parallel Corpus Augmentation using Masked Language Models

arXiv 2410.03194 ↗First posted 4 Oct 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we propose a novel method of augmenting parallel text corpora which promises good quality and is also capable of producing many fold larger corpora than the seed corpus we start with. We do not need any additional monolingual corpora. We use Multi-Lingual Masked Language Model to mask and predict alternative words in context and we use Sentence Embeddings to check and select sentence pairs which are likely to be translations of each other. We cross check our method using metrics for MT Quality Estimation. We believe this method can greatly alleviate the data scarcity problem for all language pairs for which a reasonable seed corpus is available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

25 of 40 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  7. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  11. 2017Six Challenges for Neural Machine Translationcited by 335
  12. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
13 more
  1. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  2. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 138
  3. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  4. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  5. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 87
  6. 2021AEDA: An Easier Data Augmentation Technique for Text Classificationcited by 67
  7. 2020TransQuest: Translation Quality Estimation with Cross-lingual Transformerscited by 59
  8. 2016Zero-Resource Translation with Multi-Lingual Neural Machine Translationcited by 46
  9. 2019Improving short text classification through global augmentation methodscited by 33
  10. 2018Back-Translation-Style Data Augmentation for End-to-End ASRcited by 28
  11. 2021AUGVIC: Exploiting BiText Vicinity for Low-Resource NMTcited by 4
  12. 2024Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modelingcited by 4
  13. 2023Quality Estimation of Machine Translated Texts based on Direct Evidence from Training Datacited by 1

Built on by

Not cited yet.