mlsanityData to 7 Oct 2026

Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modeling

arXiv 2401.01830 ↗First posted 3 Jan 2024cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Data augmentation is an effective technique for improving the performance of machine learning models. However, it has not been explored as extensively in natural language processing (NLP) as it has in computer vision. In this paper, we propose a novel text augmentation method that leverages the Fill-Mask feature of the transformer-based BERT model. Our method involves iteratively masking words in a sentence and replacing them with language model predictions. We have tested our proposed method on various NLP tasks and found it to be effective in many cases. Our results are presented along with a comparison to existing augmentation methods. Experimental results show that our proposed method significantly improves performance, especially on topic classification datasets.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 3 Jan 2024.

Builds on

17 of 26 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  10. 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 368
  11. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  12. 2017Data Noising as Smoothing in Neural Network Language Modelscited by 78
5 more
  1. 2021AEDA: An Easier Data Augmentation Technique for Text Classificationcited by 67
  2. 2022News Category Datasetcited by 42
  3. 2018Text Data Augmentation Made Simple By Leveraging NLP Cloud APIscited by 40
  4. 2019Improving short text classification through global augmentation methodscited by 33
  5. 2018Back-Translation-Style Data Augmentation for End-to-End ASRcited by 28

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modelingcited by 2
  2. 2024Advancing NLP Models with Strategic Text Augmentation: A Comprehensive Study of Augmentation Methods and Curriculum Strategiescited by 0
  3. 2024Parallel Corpus Augmentation using Masked Language Modelscited by 0