# Self-supervised Document Clustering Based on BERT with Data Augment

arXiv: https://arxiv.org/abs/2011.08523 · HTML: https://mlsanity.com/paper/2011.08523/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 2 of 2)

- [Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2108.11992/) (2021)
- [Towards Consistency Filtering-Free Unsupervised Learning for Dense Retrieval](https://mlsanity.com/paper/2308.02926/) (2023)

## Abstract (arXiv)

Contrastive learning is a promising approach to unsupervised learning, as it inherits the advantages of well-studied deep models without a dedicated and complex model design. In this paper, based on bidirectional encoder representations from transformers, we propose self-supervised contrastive learning (SCL) as well as few-shot contrastive learning (FCL) with unsupervised data augmentation (UDA) for text clustering. SCL outperforms state-of-the-art unsupervised clustering approaches for short texts and those for long texts in terms of several clustering evaluation measures. FCL achieves performance close to supervised learning, and FCL with UDA further improves the performance for short texts.