mlsanityData to 7 Oct 2026

Supporting Clustering with Contrastive Learning

arXiv 2103.12953 ↗First posted 24 Mar 2021cs.LG cs.CLcited by 90

Not on the map. Why?

Abstract (arXiv)

Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space. However, different categories often overlap with each other in the representation space at the beginning of the learning process, which poses a significant challenge for distance-based clustering in achieving good separation between different categories. To this end, we propose Supporting Clustering with Contrastive Learning (SCCL) -- a novel framework to leverage contrastive learning to promote better separation. We assess the performance of SCCL on short text clustering and show that SCCL significantly advances the state-of-the-art results on most benchmark datasets with 3%-11% improvement on Accuracy and 4%-15% improvement on Normalized Mutual Information. Furthermore, our quantitative analysis demonstrates the effectiveness of SCCL in leveraging the strengths of both bottom-up instance discrimination and top-down clustering to achieve better intra-cluster and inter-cluster distances when evaluated with the ground truth cluster labels.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

35 of 61 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  10. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  11. 2020Supervised Contrastive Learningcited by 2,803
  12. 2018Deep contextualized word representationscited by 2,285
23 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  4. 2016Colorful Image Colorizationcited by 1,113
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  6. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  7. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  8. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  9. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  10. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  11. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  12. 2016Variational Deep Embedding: An Unsupervised and Generative Approach to Clusteringcited by 219
  13. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  14. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  15. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  16. 2018SpectralNet: Spectral Clustering using Deep Neural Networkscited by 94
  17. 2019Mixture Models for Diverse Machine Translation: Tricks of the Tradecited by 77
  18. 2020CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understandingcited by 41
  19. 2017Self-Taught Convolutional Neural Networks for Short Text Clusteringcited by 34
  20. 2015Text Understanding from Scratchcited by 25
  21. 2020i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learningcited by 19
  22. 2020Enhancement of Short Text Clustering by Iterative Classificationcited by 16
  23. 2017Deep Unsupervised Clustering Using Mixture of Autoencoderscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 25; 2023: 30; 2024: 13; 2025: 13; 2026: 3.

90 papers.

Sort:

By area:

  1. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  2. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  3. 2022A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directionscited by 73
  4. 2023Large Language Models Enable Few-Shot Clusteringcited by 72
  5. 2023ClusterLLM: Large Language Models as a Guide for Text Clusteringcited by 69
  6. 2022Twin Contrastive Learning for Online Clusteringcited by 60
  7. 2022Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learningcited by 41
  8. 2021You Never Cluster Alonecited by 38
  9. 2022Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learningcited by 38
  10. 2022UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Miningcited by 33
  11. 2022OpenCon: Open-world Contrastive Learningcited by 33
  12. 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
38 more
  1. 2022New Intent Discovery with Pre-training and Contrastive Learningcited by 28
  2. 2023IDAS: Intent Discovery with Abstractive Summarizationcited by 22
  3. 2023When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysiscited by 21
  4. 2021Self-supervised Contrastive Attributed Graph Clusteringcited by 18
  5. 2022ContraCLM: Contrastive Learning For Causal Language Modelcited by 18
  6. 2022Towards Textual Out-of-Domain Detection without In-Domain Labelscited by 14
  7. 2023Unsupervised Representation Learning for Time Series: A Reviewcited by 14
  8. 2023Debiasing Multimodal Sarcasm Detection with Contrastive Learningcited by 12
  9. 2022Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clusteringcited by 11
  10. 2022Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representationscited by 11
  11. 2023Unsupervised Cross-Domain Rumor Detection with Contrastive Learning and Cross-Attentioncited by 11
  12. 2022Self-supervised Context-aware Style Representation for Expressive Speech Synthesiscited by 10
  13. 2023Intent Induction from Conversations for Task-Oriented Dialogue Track at DSTC 11cited by 10
  14. 2024Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Maskcited by 10
  15. 2023Towards Personalized Review Summarization by Modeling Historical Reviews from Customer and Product Separatelycited by 9
  16. 2023A Clustering Framework for Unsupervised and Semi-supervised New Intent Discoverycited by 9
  17. 2024Context-Aware Clustering using Large Language Modelscited by 9
  18. 2022Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learningcited by 8
  19. 2022Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networkscited by 7
  20. 2023Hard Regularization to Prevent Deep Online Clustering Collapse without Data Augmentationcited by 7
  21. 2023Robust Representation Learning with Reliable Pseudo-labels Generation via Self-Adaptive Optimal Transport for Short Text Clusteringcited by 6
  22. 2024A Survey of Ontology Expansion for Conversational Understandingcited by 6
  23. 2022ProtoCLIP: Prototypical Contrastive Language Image Pretrainingcited by 5
  24. 2022Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Surveycited by 5
  25. 2023Clustering-Aware Negative Sampling for Unsupervised Sentence Representationcited by 5
  26. 2023Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisationcited by 5
  27. 2024MiniConGTS: A Near Ultimate Minimalist Contrastive Grid Tagging Scheme for Aspect Sentiment Triplet Extractioncited by 5
  28. 2021MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Taggingcited by 4
  29. 2022Semi-Supervised Clustering with Contrastive Learning for Discovering New Intentscited by 4
  30. 2022Semi-supervised New Event Type Induction and Description via Contrastive Loss-Enforced Batch Attentioncited by 4
  31. 2022Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Mediacited by 4
  32. 2023Generating Faithful Text From a Knowledge Graph with Noisy Reference Textcited by 4
  33. 2023StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesiscited by 4
  34. 2025Summaries as Centroids for Interpretable and Scalable Text Clusteringcited by 4
  35. 2021Cluster Analysis with Deep Embeddings and Contrastive Learningcited by 3
  36. 2022Out-of-Dynamics Imitation Learning from Multimodal Demonstrationscited by 3
  37. 2023Relating tSNE and UMAP to Classical Dimensionality Reductioncited by 3
  38. 2024Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterancescited by 3

50 most cited of 90 All 90 (JSON)