# Unsupervised visualization of image datasets using contrastive learning

arXiv: https://arxiv.org/abs/2210.09879 · HTML: https://mlsanity.com/paper/2210.09879/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning](https://mlsanity.com/paper/2108.01775/) (2021)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [Visualizing Large-scale and High-dimensional Data](https://mlsanity.com/paper/1602.00370/) (2016)
- [Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding](https://mlsanity.com/paper/1712.09005/) (2017)
- [Do We Train on Test Data? Purging CIFAR of Near-Duplicates](https://mlsanity.com/paper/1902.00423/) (2019)
- [TriMap: Large-scale Dimensionality Reduction Using Triplets](https://mlsanity.com/paper/1910.00204/) (2019)
- [Attraction-Repulsion Spectrum in Neighbor Embeddings](https://mlsanity.com/paper/2007.08902/) (2020)
- [Understanding Deep Contrastive Learning via Coordinate-wise Optimization](https://mlsanity.com/paper/2201.12680/) (2022)
- [On UMAP's true loss function](https://mlsanity.com/paper/2103.14608/) (2021)
- [Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding](https://mlsanity.com/paper/2205.14814/) (2022)
- [From t-SNE to UMAP with contrastive learning](https://mlsanity.com/paper/2206.01816/) (2022)
- [DLME: Deep Local-flatness Manifold Embedding](https://mlsanity.com/paper/2207.03160/) (2022)
- [TLDR: Twin Learning for Dimensionality Reduction](https://mlsanity.com/paper/2110.09455/) (2021)

## Built on by (most cited first, 16 of 16)

- [Geometric Autoencoders -- What You See is What You Decode](https://mlsanity.com/paper/2306.17638/) (2023)
- [I-Con: A Unifying Framework for Representation Learning](https://mlsanity.com/paper/2504.16929/) (2025)
- [Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research](https://mlsanity.com/paper/2504.13101/) (2025)
- [Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage](https://mlsanity.com/paper/2306.03335/) (2023)
- [Navigating the Effect of Parametrization for Dimensionality Reduction](https://mlsanity.com/paper/2411.15894/) (2024)
- [AEye: A Visualization Tool for Image Datasets](https://mlsanity.com/paper/2408.04072/) (2024)
- [FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction](https://mlsanity.com/paper/2409.11509/) (2024)
- [Node Embeddings via Neighbor Embeddings](https://mlsanity.com/paper/2503.23822/) (2025)
- [MAPLE: Self-Supervised Learning-Enhanced Nonlinear Dimensionality Reduction for Visual Analysis](https://mlsanity.com/paper/2601.20173/) (2026)
- [Self-supervised Visualisation of Medical Image Datasets](https://mlsanity.com/paper/2402.14566/) (2024)
- [Examining Changes in Internal Representations of Continual Learning Models Through Tensor Decomposition](https://mlsanity.com/paper/2405.03244/) (2024)
- [Group Contrastive Learning for Weakly Paired Multimodal Data](https://mlsanity.com/paper/2602.04021/) (2026)
- [The Confusion is Real: GRAPHIC -- A Network Science Approach to Confusion Matrices in Deep Learning](https://mlsanity.com/paper/2602.19770/) (2026)
- [Towards Interpretable Foundation Models for Retinal Fundus Images](https://mlsanity.com/paper/2603.18846/) (2026)
- [The Rashomon Effect for Visualizing High-Dimensional Data](https://mlsanity.com/paper/2604.00485/) (2026)
- [CosMAP: Contrastive Manifold Approximation and Projection for Dimensionality Reduction of Omics and Genealogical Data](https://mlsanity.com/paper/2608.11269/) (2026)

## Abstract (arXiv)

Visualization methods based on the nearest neighbor graph, such as t-SNE or UMAP, are widely used for visualizing high-dimensional data. Yet, these approaches only produce meaningful results if the nearest neighbors themselves are meaningful. For images represented in pixel space this is not the case, as distances in pixel space are often not capturing our sense of similarity and therefore neighbors are not semantically close. This problem can be circumvented by self-supervised approaches based on contrastive learning, such as SimCLR, relying on data augmentation to generate implicit neighbors, but these methods do not produce two-dimensional embeddings suitable for visualization. Here, we present a new method, called t-SimCNE, for unsupervised visualization of image data. T-SimCNE combines ideas from contrastive learning and neighbor embeddings, and trains a parametric mapping from the high-dimensional pixel space into two dimensions. We show that the resulting 2D embeddings achieve classification accuracy comparable to the state-of-the-art high-dimensional SimCLR representations, thus faithfully capturing semantic relationships. Using t-SimCNE, we obtain informative visualizations of the CIFAR-10 and CIFAR-100 datasets, showing rich cluster structure and highlighting artifacts and outliers.