mlsanityData to 7 Oct 2026

Towards Good Practices in Self-supervised Representation Learning

arXiv 2012.00868 ↗First posted 1 Dec 2020cs.CV cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Self-supervised representation learning has seen remarkable progress in the last few years. More recently, contrastive instance learning has shown impressive results compared to its supervised learning counterparts. However, even with the ever increased interest in contrastive instance learning, it is still largely unclear why these methods work so well. In this paper, we aim to unravel some of the mysteries behind their success, which are the good practices. Through an extensive empirical analysis, we hope to not only provide insights but also lay out a set of best practices that led to the success of recent work in self-supervised representation learning.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 7; 2024: 1; 2025: 0; 2026: 2.

19 papers.

Sort:

  1. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  2. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 64
  3. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  4. 2022ReMix: A General and Efficient Framework for Multiple Instance Learning based Whole Slide Image Classificationcited by 30
  5. 2022Guillotine Regularization: Why removing layers is needed to improve generalization in Self-Supervised Learningcited by 25
  6. 2022Beyond Supervised vs. Unsupervised: Representative Benchmarking and Analysis of Image Representation Learningcited by 20
  7. 2023EMP-SSL: Towards Self-Supervised Learning in One Training Epochcited by 18
  8. 2022Toward a Geometrical Understanding of Self-supervised Contrastive Learningcited by 17
  9. 2023All4One: Symbiotic Neighbour Contrastive Learning via Self-Attention and Redundancy Reductioncited by 11
  10. 2021An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysiscited by 10
  11. 2022Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representationscited by 9
  12. 2024Learning Data Association for Multi-Object Tracking using Only Coordinatescited by 8
7 more
  1. 2023Evaluating Self-Supervised Learning via Risk Decompositioncited by 6
  2. 2023Deciphering the Projection Head: Representation Evaluation Self-supervised Learningcited by 5
  3. 2023Data Compression and Inference in Cosmology with Self-Supervised Machine Learningcited by 5
  4. 2022Temporal Contrastive Learning with Curriculumcited by 1
  5. 2023Towards Unsupervised Representation Learning: Learning, Evaluating and Transferring Visual Representationscited by 0
  6. 2026Show Me What You Don't Know: Efficient Sampling from Invariant Sets for Model Validationcited by 0
  7. 2026Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptationcited by 0