mlsanityData to 14 Oct 2026

Tailoring Self-Supervision for Supervised Learning

arXiv 2207.10023 ↗First posted 20 Jul 2022cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Recently, it is shown that deploying a proper self-supervision is a prospective way to enhance the performance of supervised learning. Yet, the benefits of self-supervision are not fully exploited as previous pretext tasks are specialized for unsupervised representation learning. To this end, we begin by presenting three desirable properties for such auxiliary tasks to assist the supervised objective. First, the tasks need to guide the model to learn rich features. Second, the transformations involved in the self-supervision should not significantly alter the training distribution. Third, the tasks are preferred to be light and generic for high applicability to prior arts. Subsequently, to show how existing pretext tasks can fulfill these and be tailored for supervised learning, we propose a simple auxiliary self-supervision task, predicting localizable rotation (LoRot). Our exhaustive experiments validate the merits of LoRot as a pretext task tailored for supervised learning in terms of robustness and generalization capability. Our code is available at https://github.com/wjun0830/Localizable-Rotation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 20 Jul 2022.

Builds on

55 of 70 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  8. 2013Intriguing properties of neural networkscited by 5,155
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2016Wide Residual Networkscited by 3,070
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
38 more
  1. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  2. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  3. 2020Supervised Contrastive Learningcited by 2,829
  4. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  7. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  8. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  12. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  13. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  14. 2017Random Erasing Data Augmentationcited by 1,239
  15. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  16. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  17. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  18. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  19. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  20. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  21. 2017Focal Loss for Dense Object Detectioncited by 752
  22. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
  23. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  24. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  25. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  26. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 300
  27. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 256
  28. 2019Fast AutoAugmentcited by 243
  29. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 232
  30. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  31. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 213
  32. 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
  33. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  34. 2018Adversarial Complementary Learning for Weakly Supervised Object Localizationcited by 166
  35. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  36. 2021Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversitycited by 128
  37. 2020Learning Open Set Network with Discriminative Reciprocal Pointscited by 115
  38. 2018Boosting Self-Supervised Learning via Knowledge Transfercited by 92

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 2.

7 papers.

  1. 2023Leveraging Hidden Positives for Unsupervised Semantic Segmentationcited by 40
  2. 2025Selective Contrastive Learning for Weakly Supervised Affordance Groundingcited by 6
  3. 2022Minority-Oriented Vicinity Expansion with Attentive Aggregation for Video Long-Tailed Recognitioncited by 3
  4. 2023Gated Self-supervised Learning For Improving Supervised Learningcited by 1
  5. 2025Dual-supervised Asymmetric Co-training for Semi-supervised Medical Domain Generalizationcited by 1
  6. 2023Mixture of Self-Supervised Learningcited by 0
  7. 2024Extremely Fine-Grained Visual Classification over Resembling Glyphs in the Wildcited by 0