Unsupervised hard Negative Augmentation for contrastive learning
Not on the map. Why?
Abstract (arXiv)
We present Unsupervised hard Negative Augmentation (UNA), a method that generates synthetic negative instances based on the term frequency-inverse document frequency (TF-IDF) retrieval model. UNA uses TF-IDF scores to ascertain the perceived importance of terms in a sentence and then produces negative samples by replacing terms with respect to that. Our experiments demonstrate that models trained with UNA improve the overall performance in semantic textual similarity tasks. Additional performance gains are obtained when combining UNA with the paraphrasing augmentation. Further results show that our method is compatible with different backbone models. Ablation studies also support the choice of having a TF-IDF-driven control on negative augmentation.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 5 Jan 2024.
Builds on
27 of 42 references.
References · 27
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
- 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
15 more
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
- 2020Contrastive Learning with Hard Negative Samplescited by 619
- 2020Hard Negative Mixing for Contrastive Learningcited by 399
- 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
- 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
- 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
- 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
- 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
- 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
- 2021AEDA: An Easier Data Augmentation Technique for Text Classificationcited by 67
- 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
- 2021Negative Data Augmentationcited by 46
- 2022SCD: Self-Contrastive Decorrelation for Sentence Embeddingscited by 19
- 2021Efficient Contrastive Learning via Novel Data Augmentation and Curriculum Learningcited by 14
- 2022AugCSE: Contrastive Sentence Embedding with Diverse Augmentationscited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.