mlsanityData to 14 Oct 2026

Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty

arXiv 1906.12340 ↗First posted 28 Jun 2019cs.LG cs.CV stat.MLcited by 514

Not on the map. Why?

Abstract (arXiv)

Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often not thought beneficial beyond obviating or reducing the need for annotations. We find that self-supervision can benefit robustness in a variety of ways, including robustness to adversarial examples, label corruption, and common input corruptions. Additionally, self-supervision greatly benefits out-of-distribution detection on difficult, near-distribution outliers, so much so that it exceeds the performance of fully supervised methods. These results demonstrate the promise of self-supervision for improving robustness and uncertainty estimation and establish these tasks as new axes of evaluation for future self-supervised learning research.

Key facts. Cited by 514 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 118; 2022: 131; 2023: 128; 2024: 65; 2025: 55; 2026: 17.

514 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  3. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  4. 2021Unsolved Problems in ML Safetycited by 292
  5. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  6. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
  7. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  8. 2023A Cookbook of Self-Supervised Learningcited by 250
  9. 2023Pretraining Language Models with Human Preferencescited by 247
  10. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
  11. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  12. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
38 more
  1. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  2. 2022Backdoor Defense via Decoupling the Training Processcited by 153
  3. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 148
  4. 2021Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesiscited by 140
  5. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 132
  6. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  7. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  8. 2023GEO-Bench: Toward Foundation Models for Earth Monitoringcited by 124
  9. 2021Predicting with Confidence on Unseen Distributionscited by 116
  10. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  11. 2021Neural Transformation Learning for Deep Anomaly Detection Beyond Imagescited by 99
  12. 2022Visual Prompt Tuning for Test-time Domain Adaptationcited by 97
  13. 2022Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detectioncited by 93
  14. 2021Mean-Shifted Contrastive Loss for Anomaly Detectioncited by 88
  15. 2021Adversarially Adaptive Normalization for Single Domain Generalizationcited by 85
  16. 2022A Theoretical Study on Solving Continual Learningcited by 80
  17. 2021Self-Supervised Learning with Kernel Dependence Maximizationcited by 77
  18. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 77
  19. 2022Spatio-Temporal Self-Supervised Learning for Traffic Flow Predictioncited by 77
  20. 2022Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantificationcited by 73
  21. 2022Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detectioncited by 72
  22. 2021Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learningcited by 68
  23. 2022Audio Self-supervised Learning: A Surveycited by 65
  24. 2021Understanding Softmax Confidence and Uncertaintycited by 63
  25. 2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
  26. 2022Video Transformers: A Surveycited by 63
  27. 2022Video Anomaly Detection by Solving Decoupled Spatio-Temporal Jigsaw Puzzlescited by 62
  28. 2022X-Risk Analysis for AI Researchcited by 60
  29. 2022Normalizing Flows for Human Pose Anomaly Detectioncited by 60
  30. 2023Unsupervised Out-of-Distribution Detection with Diffusion Inpaintingcited by 59
  31. 2021Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articlescited by 58
  32. 2021NGC: A Unified Framework for Learning with Open-World Noisy Datacited by 58
  33. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  34. 2023Anomaly Detection with Conditioned Denoising Diffusion Modelscited by 56
  35. 2021A Survey of Machine Learning for Computer Architecture and Systemscited by 53
  36. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  37. 2024Community Forensics: Using Thousands of Generators to Train Fake Image Detectorscited by 50
  38. 2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49

50 most cited of 514 All 514 (JSON)