mlsanityData to 7 Oct 2026

DenseDINO: Boosting Dense Self-Supervised Learning with Token-Based Point-Level Consistency

arXiv 2306.04654 ↗First posted 6 Jun 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a simple yet effective transformer framework for self-supervised learning called DenseDINO to learn dense visual representations. To exploit the spatial information that the dense prediction tasks require but neglected by the existing self-supervised transformers, we introduce point-level supervision across views in a novel token-based way. Specifically, DenseDINO introduces some extra input tokens called reference tokens to match the point-level features with the position prior. With the reference token, the model could maintain spatial consistency and deal with multi-object complex scene images, thus generalizing better on dense prediction tasks. Compared with the vanilla DINO, our approach obtains competitive performance when evaluated on classification in ImageNet and achieves a large margin (+7.2% mIoU) improvement in semantic segmentation on PascalVOC under the linear probing protocol for segmentation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

22 of 27 references.

References · 22

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
10 more
  1. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  2. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  3. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  4. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  5. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  6. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  7. 2020Self-EMD: Self-Supervised Object Detection without ImageNetcited by 70
  8. 2022Point-Level Region Contrast for Object Detection Pre-Trainingcited by 41
  9. 2022Learning Where to Learn in Cross-View Self-Supervised Learningcited by 31
  10. 2022Self-supervised Semantic Segmentation Grounded in Visual Conceptscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixingcited by 27
  2. 2025Perceptual Inductive Bias Is What You Need Before Contrastive Learningcited by 4
  3. 2025Investigating Location-Regularised Self-Supervised Feature Learning for Seafloor Visual Imagerycited by 2
  4. 2025ACE: Anatomically Consistent Embeddings in Composition and Decompositioncited by 1
  5. 2025A Comparative Study of Vision Transformers and CNNs for Few-Shot Rigid Transformation and Fundamental Matrix Estimationcited by 0
  6. 2026Suppressing Non-Semantic Noise in Masked Image Modeling Representationscited by 0