mlsanityData to 14 Oct 2026

Which Tasks Survive Self-Supervised Learning?

arXiv 2609.38393 ↗First posted 29 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Same-instance self-supervised learning (SSL) learns representations by enforcing consistency across two views of the same underlying instance. This principle alone, however, does not determine which downstream tasks remain recoverable from the learned representation. We study this question through semantic recoverability, defined as the amount of a task's posterior score captured by the represented function space. We show that, for centered and whitened representations, recoverability exactly determines directional class-distance-normalized variance (CDNV), controls few-shot nearest-centroid classification, and governs the strength of task-relevant semantic directions. The population linear probe and centroid axis coincide, and multiple well-recovered tasks approach a factorial centroid geometry. We then analyze a canonical two-view SSL objective and show that its population optimum spans the leading cross-view-stable modes of the associated two-view operator. This yields a closed-form spectral characterization of semantic recoverability: a downstream task is preserved to the extent that its posterior lies in the selected spectral subspace. We validate these predictions on synthetic and real datasets across several SSL methods, testing the predicted relationships among recoverability, directional geometry, spectral structure, and few-shot transfer. Together, these results give a task-level account of what information survives same-instance SSL and how the retained information appears in downstream geometry and transfer.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 61 of its 76 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

61 of 76 references.

References · 61

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
38 more
  1. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  5. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  11. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  12. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  13. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  14. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  15. 2020Understanding the Behaviour of Contrastive Losscited by 398
  16. 2019On Mutual Information Maximization for Representation Learningcited by 316
  17. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  18. 2020Whitening for Self-Supervised Representation Learningcited by 279
  19. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  20. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  21. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 158
  22. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
  23. 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
  24. 2020Intriguing Properties of Contrastive Lossescited by 130
  25. 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
  26. 2022On the duality between contrastive and non-contrastive self-supervised learningcited by 108
  27. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  28. 2022Understanding Contrastive Learning Requires Incorporating Inductive Biasescited by 96
  29. 2022Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptationcited by 67
  30. 2021Investigating the Role of Negatives in Contrastive Representation Learningcited by 54
  31. 2021Exploring the Equivalence of Siamese Self-Supervised Learning via A Unified Gradient Frameworkcited by 49
  32. 2020Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Taskscited by 47
  33. 2021Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learningcited by 47
  34. 2022A Theoretical Study of Inductive Biases in Contrastive Learningcited by 40
  35. 2022Do More Negative Samples Necessarily Hurt in Contrastive Learning?cited by 39
  36. 2023InfoNCE Loss Provably Learns Cluster-Preserving Representationscited by 37
  37. 2022Understanding and Improving the Role of Projection Head in Self-Supervised Learningcited by 36
  38. 2023Reverse Engineering Self-Supervised Learningcited by 33

50 most cited of 61 All 61 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning
  2. 2026Self-Supervised Learning from Structural Invariance
  3. 2022Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing
  4. 2025Bound by semanticity: universal laws governing the generalization-identification tradeoff
  5. 2026Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning
  6. 2025Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Models
  7. 2024Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?
  8. 2026Reflective Recovery: A Self-Supervised Method for Reasoning by Learning from Mistakes