mlsanityData to 14 Oct 2026

Probabilistic Linguistic Knowledge and Token-level Text Augmentation

arXiv 2306.16644 ↗First posted 29 Jun 2023cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper investigates the effectiveness of token-level text augmentation and the role of probabilistic linguistic knowledge within a linguistically-motivated evaluation context. Two text augmentation programs, REDA and REDANG, were developed, both implementing five token-level text editing operations: Synonym Replacement (SR), Random Swap (RS), Random Insertion (RI), Random Deletion (RD), and Random Mix (RM). REDANG leverages pretrained n-gram language models to select the most likely augmented texts from REDA's output. Comprehensive and fine-grained experiments were conducted on a binary question matching classification task in both Chinese and English. The results strongly refute the general effectiveness of the five token-level text augmentation techniques under investigation, whether applied together or separately, and irrespective of various common classification model types used, including transformers. Furthermore, the role of probabilistic linguistic knowledge is found to be minimal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 48 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

26 of 48 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  8. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  9. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  11. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  12. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 261
14 more
  1. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  2. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
  3. 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
  4. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  5. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 78
  6. 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 68
  7. 2020How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers?cited by 41
  8. 2019XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answeringcited by 35
  9. 2020ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understandingcited by 17
  10. 2018AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examplescited by 15
  11. 2021To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLPcited by 12
  12. 2022DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classificationcited by 10
  13. 2021Linguistic Knowledge in Data Augmentation for Natural Language Processing: An Example on Chinese Question Matchingcited by 1
  14. 2022Random Text Perturbations Work, but not Alwayscited by 1

Built on by

Not cited yet.