mlsanityData to 14 Oct 2026

LINDA: Unsupervised Learning to Interpolate in Natural Language Processing

arXiv 2112.13969 ↗First posted 28 Dec 2021cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite the success of mixup in data augmentation, its applicability to natural language processing (NLP) tasks has been limited due to the discrete and variable-length nature of natural languages. Recent studies have thus relied on domain-specific heuristics and manually crafted resources, such as dictionaries, in order to apply mixup in NLP. In this paper, we instead propose an unsupervised learning approach to text interpolation for the purpose of data augmentation, to which we refer as "Learning to INterpolate for Data Augmentation" (LINDA), that does not require any heuristics nor manually crafted resources but learns to interpolate between any pair of natural language sentences over a natural language manifold. After empirically demonstrating the LINDA's interpolation capability, we show that LINDA indeed allows us to seamlessly apply mixup in NLP and leads to better generalization in text classification both in-domain and out-of-domain.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 28 of its 39 references that are in the map. First posted on arXiv 28 Dec 2021.

Builds on

28 of 39 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  10. 2013Auto-Encoding Variational Bayescited by 4,053
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
16 more
  1. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  2. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  3. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  4. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  5. 2019Fast AutoAugmentcited by 243
  6. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
  7. 2019Not Enough Data? Deep Learning to the Rescue!cited by 175
  8. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  9. 2019Augmenting Data with Mixup for Sentence Classification: An Empirical Studycited by 125
  10. 2018Conditional BERT Contextual Augmentationcited by 107
  11. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 96
  12. 2015Reasoning about Entailment with Neural Attentioncited by 69
  13. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  14. 2021SSMix: Saliency-Based Span Mixup for Text Classificationcited by 39
  15. 2021Reweighting Augmented Samples by Minimizing the Maximal Expected Losscited by 21
  16. 2020Mixup-Transformer: Dynamic Data Augmentation for NLP Taskscited by 15

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 2.

6 papers.

  1. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  2. 2023PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillationcited by 25
  3. 2022DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classificationcited by 10
  4. 2024AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimescited by 1
  5. 2024Enhancing Effectiveness and Robustness in a Low-Resource Regime via Decision-Boundary-aware Data Augmentationcited by 1
  6. 2023Blind Biological Sequence Denoising with Self-Supervised Set Learningcited by 0