mlsanityData to 7 Oct 2026

Conditional BERT Contextual Augmentation

arXiv 1812.06705 ↗First posted 17 Dec 2018cs.CL cs.AI cs.LGcited by 107

Not on the map. Why?

Abstract (arXiv)

We propose a novel data augmentation method for labeled sentences called conditional BERT contextual augmentation. Data augmentation methods are often applied to prevent overfitting and improve generalization of deep neural network models. Recently proposed contextual augmentation augments labeled sentences by randomly replacing words with more varied substitutions predicted by language model. BERT demonstrates that a deep bidirectional language model is more powerful than either an unidirectional language model or the shallow concatenation of a forward and backward model. We retrofit BERT to conditional BERT by introducing a new conditional masked language modelfootnoteThe term "conditional masked language model" appeared once in original BERT paper, which indicates context-conditional, is equivalent to term "masked language model". In our paper, "conditional masked language model" indicates we apply extra label-conditional constraint to the "masked language model". task. The well trained conditional BERT can be applied to enhance contextual augmentation. Experiments on six various different text classification tasks show that our method can be easily applied to both convolutional or recurrent neural networks classifier to obtain obvious improvement.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 27; 2023: 25; 2024: 19; 2025: 6; 2026: 3.

107 papers.

Sort:

By area:

  1. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  2. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  3. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  4. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 73
  5. 2021AEDA: An Easier Data Augmentation Technique for Text Classificationcited by 67
  6. 2021Text Detoxification using Large Pre-trained Neural Modelscited by 52
  7. 2022PromDA: Prompt-based Data Augmentation for Low-Resource NLU Taskscited by 51
  8. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  9. 2021Generate, Annotate, and Learn: NLP with Synthetic Textcited by 42
  10. 2021MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NERcited by 42
  11. 2022Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimationcited by 38
  12. 2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
38 more
  1. 2023ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPTcited by 32
  2. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  3. 2023PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillationcited by 25
  4. 2021GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentationcited by 20
  5. 2023Using GPT-4 to Augment Unbalanced Data for Automatic Scoringcited by 20
  6. 2024Empowering Large Language Models for Textual Data Augmentationcited by 17
  7. 2021Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classificationcited by 14
  8. 2022PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentationcited by 14
  9. 2023Self-supervised Representation Learning From Random Data Projectorscited by 14
  10. 2021Methods for Detoxification of Texts for the Russian Languagecited by 13
  11. 2022Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Taskscited by 13
  12. 2022A Feature-space Multimodal Data Augmentation Technique for Text-video Retrievalcited by 13
  13. 2021To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLPcited by 12
  14. 2022A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysiscited by 12
  15. 2022Fact Checking with Insufficient Evidencecited by 11
  16. 2022GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentationcited by 11
  17. 2021Consistency Training with Virtual Adversarial Discrete Perturbationcited by 9
  18. 2021Few-shot learning through contextual data augmentationcited by 8
  19. 2022BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representationcited by 8
  20. 2024LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs?cited by 8
  21. 2021Data Augmentation for Mental Health Classification on Social Mediacited by 7
  22. 2022PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Taskscited by 7
  23. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  24. 2021LINDA: Unsupervised Learning to Interpolate in Natural Language Processingcited by 6
  25. 2022Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translationcited by 6
  26. 2024A Large Language Model and Denoising Diffusion Framework for Targeted Design of Microstructures with Commands in Natural Languagecited by 6
  27. 2021Unsupervised Document Embedding via Contrastive Augmentationcited by 5
  28. 2022Semantic-based Data Augmentation for Math Word Problemscited by 5
  29. 2022When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentationcited by 5
  30. 2022Data Augmentation for Biomedical Factoid Question Answeringcited by 5
  31. 2022Disambiguation of morpho-syntactic features of African American English -- the case of habitual becited by 5
  32. 2023Target-Side Augmentation for Document-Level Machine Translationcited by 5
  33. 2023Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Taskscited by 5
  34. 2024Iterative Data Generation with Large Language Models for Aspect-based Sentiment Analysiscited by 5
  35. 2024An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classificationcited by 5
  36. 2025Balanced Training Data Augmentation for Aspect-Based Sentiment Analysiscited by 5
  37. 2021Data Augmentation for Text Generation Without Any Augmented Datacited by 4
  38. 2021AUGVIC: Exploiting BiText Vicinity for Low-Resource NMTcited by 4

50 most cited of 107 All 107 (JSON)