# Combining pretrained CNN feature extractors to enhance clustering of complex natural images

arXiv: https://arxiv.org/abs/2101.02767 · HTML: https://mlsanity.com/paper/2101.02767/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 69 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [Learning Discrete Representations via Information Maximizing Self-Augmented Training](https://mlsanity.com/paper/1702.08720/) (2017)
- [Discriminatively Boosted Image Clustering with Fully Convolutional Auto-Encoders](https://mlsanity.com/paper/1703.07980/) (2017)
- [Deep Density-based Image Clustering](https://mlsanity.com/paper/1812.04287/) (2018)
- [Differentiable Deep Clustering with Cluster Size Constraints](https://mlsanity.com/paper/1910.09036/) (2019)
- [Iterative Views Agreement: An Iterative Low-Rank based Structured Optimization Method to Multi-View Spectral Clustering](https://mlsanity.com/paper/1608.05560/) (2016)
- [Improving Image Clustering With Multiple Pretrained CNN Feature Extractors](https://mlsanity.com/paper/1807.07760/) (2018)
- [CNN features are also great at unsupervised classification](https://mlsanity.com/paper/1707.01700/) (2017)
- [Multi-Modal Deep Clustering: Unsupervised Partitioning of Images](https://mlsanity.com/paper/1912.02678/) (2019)
- [Unsupervised Joint Mining of Deep Features and Image Labels for Large-scale Radiology Image Categorization and Scene Recognition](https://mlsanity.com/paper/1701.06599/) (2017)
- [SA-Net: A deep spectral analysis network for image clustering](https://mlsanity.com/paper/2009.07026/) (2020)
- [Unsupervised robotic sorting: Towards autonomous decision making robots](https://mlsanity.com/paper/1804.04572/) (2018)

## Built on by (most cited first, 7 of 7)

- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [Iterative Cluster Harvesting for Wafer Map Defect Patterns](https://mlsanity.com/paper/2404.15436/) (2024)
- [Vision Backbone Efficient Selection for Image Classification in Low-Data Regimes](https://mlsanity.com/paper/2410.08592/) (2024)
- [Dataset Summarization by K Principal Concepts](https://mlsanity.com/paper/2104.03952/) (2021)
- [Seeded iterative clustering for histology region identification](https://mlsanity.com/paper/2211.07425/) (2022)
- [A Personalized Video-Based Hand Taxonomy: Application for Individuals with Spinal Cord Injury](https://mlsanity.com/paper/2403.18094/) (2024)
- [ELMF4EggQ: Ensemble Learning with Multimodal Feature Fusion for Non-Destructive Egg Quality Assessment](https://mlsanity.com/paper/2510.02876/) (2025)

## Abstract (arXiv)

Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering. These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue, we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different architectures as different "views" of the same data. This approach is based on the assumption that information contained in the different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets, and produces state-of-the-art results for IC.