Semantic Concentration for Self-Supervised Dense Representations Learning
Not on the map. Why?
Abstract (arXiv)
Recent advances in image-level self-supervised learning (SSL) have made significant progress, yet learning dense representations for patches remains challenging. Mainstream methods encounter an over-dispersion phenomenon that patches from the same instance/category scatter, harming downstream performance on dense tasks. This work reveals that image-level SSL avoids over-dispersion by involving implicit semantic concentration. Specifically, the non-strict spatial alignment ensures intra-instance consistency, while shared patterns, i.e., similar parts of within-class instances in the input space, ensure inter-image consistency. Unfortunately, these approaches are infeasible for dense SSL due to their spatial sensitivity and complicated scene-centric data. These observations motivate us to explore explicit semantic concentration for dense SSL. First, to break the strict spatial alignment, we propose to distill the patch correspondences. Facing noisy and imbalanced pseudo labels, we propose a noise-tolerant ranking loss. The core idea is extending the Average Precision (AP) loss to continuous targets, such that its decision-agnostic and adaptive focusing properties prevent the student model from being misled. Second, to discriminate the shared patterns from complicated scenes, we propose the object-aware filter to map the output space to an object-based space. Specifically, patches are represented by learnable prototypes of objects via cross-attention. Last but not least, empirical studies across various tasks soundly support the effectiveness of our method. Code is available in https://github.com/KID-7391/CoTAP.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 64 of its 80 references that are in the map. First posted on arXiv 11 Sep 2025.
Builds on
64 of 80 references.
References · 64
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2017Mask R-CNNcited by 7,277
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
38 more
- 2020Exploring Simple Siamese Representation Learningcited by 2,866
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
- 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
- 2021Transformers in Vision: A Surveycited by 1,013
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
- 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
- 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
- 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
- 2019Self-labelling via simultaneous clustering and representation learningcited by 528
- 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 429
- 2018Rethinking ImageNet Pre-trainingcited by 377
- 2021Deep ViT Features as Dense Visual Descriptorscited by 330
- 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
- 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 256
- 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
- 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
- 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
- 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
- 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
- 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
- 2020Unsupervised Learning of Dense Visual Representationscited by 147
- 2021Self-Supervised Learning with Swin Transformerscited by 122
- 2022On the duality between contrastive and non-contrastive self-supervised learningcited by 108
- 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
- 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
- 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
- 2019Unsupervised Deep Learning by Neighbourhood Discoverycited by 78
- 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
- 2022Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?cited by 67
- 2022The Hidden Uniform Cluster Prior in Self-Supervised Learningcited by 58
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2025: 3.
3 papers.