# On the Discriminability of Self-Supervised Representation Learning

arXiv: https://arxiv.org/abs/2407.13541 · HTML: https://mlsanity.com/paper/2407.13541/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2024.

## Built on by (most cited first, 2 of 2)

- [Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanisms](https://mlsanity.com/paper/2409.07989/) (2024)
- [Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models](https://mlsanity.com/paper/2607.09785/) (2026)

## Abstract (arXiv)

Self-supervised learning (SSL) has recently shown notable success in various visual tasks. However, in terms of discriminability, SSL is still not on par with supervised learning (SL). This paper identifies a key issue, the ``crowding problem," where features from different classes are not well-separated, and there is high intra-class variance. In contrast, SL ensures clear class separation. Our analysis reveals that SSL objectives do not adequately constrain the relationships between samples and their augmentations, leading to poorer performance in complex tasks. We further establish a theoretical framework that connects SSL objectives to cross-entropy risk bounds, explaining how reducing intra-class variance and increasing inter-class separation can improve generalization. To address this, we propose the Dynamic Semantic Adjuster (DSA), a learnable regulator that enhances feature aggregation and separation while being robust to outliers. Comprehensive experiments conducted on diverse benchmark datasets validate that DSA leads to substantial gains in SSL performance, narrowing the performance gap with SL.