mlsanityData to 7 Oct 2026

Data Noising as Smoothing in Neural Network Language Models

arXiv 1703.02573 ↗First posted 7 Mar 2017cs.LG cs.CLcited by 78

Not on the map. Why?

Abstract (arXiv)

Data noising is an effective technique for regularizing neural network models. While noising is widely adopted in application domains such as vision and speech, commonly used noising primitives have not been developed for discrete sequence-level settings such as language modeling. In this paper, we derive a connection between input noising in neural network language models and smoothing in n-gram models. Using this connection, we draw upon ideas from smoothing to develop effective noising schemes. We demonstrate performance gains when applying the proposed schemes to language modeling and machine translation. Finally, we provide empirical analysis validating the relationship between noising and smoothing.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 19; 2023: 10; 2024: 10; 2025: 6; 2026: 5.

78 papers.

Sort:

By area:

  1. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  2. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  3. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  4. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
  5. 2021AEDA: An Easier Data Augmentation Technique for Text Classificationcited by 67
  6. 2024A Comprehensive Survey on Data Augmentationcited by 38
  7. 2024Leveraging LLM-Respondents for Item Evaluation: a Psychometric Analysiscited by 31
  8. 2021GNN-LM: Language Modeling based on Global Contexts via GNNcited by 27
  9. 2025Pre-training under infinite computecited by 24
  10. 2021Factorising Meaning and Form for Intent-Preserving Paraphrasingcited by 22
  11. 2021Reweighting Augmented Samples by Minimizing the Maximal Expected Losscited by 21
  12. 2022Hierarchical Sketch Induction for Paraphrase Generationcited by 21
38 more
  1. 2021Layer-wise Model Pruning based on Mutual Informationcited by 19
  2. 2021Rethinking Perturbations in Encoder-Decoders for Fast Trainingcited by 17
  3. 2024Predictive Dynamic Fusioncited by 17
  4. 2021OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contextscited by 16
  5. 2022TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understandingcited by 15
  6. 2021Phrase-level Active Learning for Neural Machine Translationcited by 13
  7. 2022Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translationcited by 13
  8. 2022A Feature-space Multimodal Data Augmentation Technique for Text-video Retrievalcited by 13
  9. 2021Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMaxcited by 10
  10. 2021Towards Efficiently Diversifying Dialogue Generation via Embedding Augmentationcited by 9
  11. 2021Consistency Training with Virtual Adversarial Discrete Perturbationcited by 9
  12. 2022CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translationcited by 9
  13. 2021Text Augmentation in a Multi-Task Viewcited by 8
  14. 2021BERT-based Acronym Disambiguation with Multiple Training Strategiescited by 8
  15. 2021Neural model robustness for skill routing in large-scale conversational AI systems: A design choice explorationcited by 8
  16. 2021Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labelingcited by 8
  17. 2021Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approachcited by 8
  18. 2022BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representationcited by 8
  19. 2024Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classificationcited by 8
  20. 2021Influence-guided Data Augmentation for Neural Tensor Completioncited by 7
  21. 2022EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classificationcited by 7
  22. 2022miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddingscited by 7
  23. 2021N-Shot Learning for Augmenting Task-Oriented Dialogue State Trackingcited by 6
  24. 2023Attributable and Scalable Opinion Summarizationcited by 6
  25. 2024A Little Human Data Goes A Long Waycited by 6
  26. 2022Semantic-based Data Augmentation for Math Word Problemscited by 5
  27. 2022Selective Text Augmentation with Word Roles for Low-Resource Text Classificationcited by 5
  28. 2025Balanced Training Data Augmentation for Aspect-Based Sentiment Analysiscited by 5
  29. 2021Data Augmentation for Text Generation Without Any Augmented Datacited by 4
  30. 2021AUGVIC: Exploiting BiText Vicinity for Low-Resource NMTcited by 4
  31. 2022AdMix: A Mixed Sample Data Augmentation Method for Neural Machine Translationcited by 4
  32. 2022Global Mixup: Eliminating Ambiguity with Clusteringcited by 3
  33. 2022Semantically Consistent Data Augmentation for Neural Machine Translation via Conditional Masked Language Modelcited by 3
  34. 2022Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Modelscited by 3
  35. 2024Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modelingcited by 3
  36. 2025Sarang at DEFACTIFY 4.0: Detecting AI-Generated Text Using Noised Data and an Ensemble of DeBERTa Modelscited by 3
  37. 2021Learning to Augment for Data-Scarce Domain BERT Knowledge Distillationcited by 2
  38. 2021Data Augmentation for Opcode Sequence Based Malware Detectioncited by 2

50 most cited of 78 All 78 (JSON)