mlsanityData to 7 Oct 2026

AugCSE: Contrastive Sentence Embedding with Diverse Augmentations

arXiv 2210.13749 ↗First posted 20 Oct 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Data augmentation techniques have been proven useful in many applications in NLP fields. Most augmentations are task-specific, and cannot be used as a general-purpose tool. In our work, we present AugCSE, a unified framework to utilize diverse sets of data augmentations to achieve a better, general purpose, sentence embedding model. Building upon the latest sentence embedding models, our approach uses a simple antagonistic discriminator that differentiates the augmentation types. With the finetuning objective borrowed from domain adaptation, we show that diverse augmentations, which often lead to conflicting contrastive signals, can be tamed to produce a better and more robust sentence representation. Our methods achieve state-of-the-art results on downstream transfer tasks and perform competitively on semantic textual similarity tasks, using only unsupervised data.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 81 of its 122 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

81 of 122 references.

References · 81

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
38 more
  1. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  3. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  9. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  10. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  11. 2018Neural Network Acceptability Judgmentscited by 952
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  13. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  14. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  15. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  16. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  17. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  18. 2015Generating Sentences from a Continuous Spacecited by 607
  19. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  20. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  21. 2017Generative Adversarial Networks: An Overviewcited by 588
  22. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  23. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  24. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  25. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  26. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  27. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  28. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  29. 2018Gender Bias in Neural Natural Language Processingcited by 210
  30. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 210
  31. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
  32. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 186
  33. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  34. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
  35. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  36. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  37. 2019Good-Enough Compositional Data Augmentationcited by 158
  38. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 1.

5 papers.

  1. 2022AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentationcited by 3
  2. 2023xSIM++: An Improved Proxy to Bitext Mining Performance for Low-Resource Languagescited by 3
  3. 2024Unsupervised hard Negative Augmentation for contrastive learningcited by 2
  4. 2024Event-enhanced Retrieval in Real-time Searchcited by 1
  5. 2025Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?cited by 1