# Improve Contrastive Clustering Performance by Multiple Fusing-Augmenting ViT Blocks

arXiv: https://arxiv.org/abs/2511.08883 · HTML: https://mlsanity.com/paper/2511.08883/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 12 Nov 2025.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation](https://mlsanity.com/paper/2106.00131/) (2021)
- [Improving Visual Prompt Tuning for Self-supervised Vision Transformers](https://mlsanity.com/paper/2306.05067/) (2023)
- [Ultra-Scalable Spectral Clustering and Ensemble Clustering](https://mlsanity.com/paper/1903.01057/) (2019)
- [Semantic-Enhanced Image Clustering](https://mlsanity.com/paper/2208.09849/) (2022)
- [CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection](https://mlsanity.com/paper/2204.05515/) (2022)
- [Toward Multidiversified Ensemble Clustering of High-Dimensional Data: From Subspaces to Metrics and Beyond](https://mlsanity.com/paper/1710.03113/) (2017)
- [Understand and Improve Contrastive Learning Methods for Visual Representation: A Review](https://mlsanity.com/paper/2106.03259/) (2021)
- [Deeply-Fused Nets](https://mlsanity.com/paper/1605.07716/) (2016)
- [Accelerated Fuzzy C-Means Clustering Based on New Affinity Filtering and Membership Scaling](https://mlsanity.com/paper/2302.07060/) (2023)

## Abstract (arXiv)

In the field of image clustering, the widely used contrastive learning networks improve clustering performance by maximizing the similarity between positive pairs and the dissimilarity of negative pairs of the inputs. Extant contrastive learning networks, whose two encoders often implicitly interact with each other by parameter sharing or momentum updating, may not fully exploit the complementarity and similarity of the positive pairs to extract clustering features from input data. To explicitly fuse the learned features of positive pairs, we design a novel multiple fusing-augmenting ViT blocks (MFAVBs) based on the excellent feature learning ability of Vision Transformers (ViT). Firstly, two preprocessed augmentions as positive pairs are separately fed into two shared-weight ViTs, then their output features are fused to input into a larger ViT. Secondly, the learned features are split into a pair of new augmented positive samples and passed to the next FAVBs, enabling multiple fusion and augmention through MFAVBs operations. Finally, the learned features are projected into both instance-level and clustering-level spaces to calculate the cross-entropy loss, followed by parameter updates by backpropagation to finalize the training process. To further enhance ability of the model to distinguish between similar images, our input data for the network we propose is preprocessed augmentions with features extracted from the CLIP pretrained model. Our experiments on seven public datasets demonstrate that MFAVBs serving as the backbone for contrastive clustering outperforms the state-of-the-art techniques in terms of clustering performance.