Self-Supervised Learning Using Nonlinear Dependence
Not on the map. Why?
Abstract (arXiv)
Self-supervised learning has gained significant attention in contemporary applications, particularly due to the scarcity of labeled data. While existing SSL methodologies primarily address feature variance and linear correlations, they often neglect the intricate relations between samples and the nonlinear dependencies inherent in complex data--especially prevalent in high-dimensional visual data. In this paper, we introduce Correlation-Dependence Self-Supervised Learning (CDSSL), a novel framework that unifies and extends existing SSL paradigms by integrating both linear correlations and nonlinear dependencies, encapsulating sample-wise and feature-wise interactions. Our approach incorporates the Hilbert-Schmidt Independence Criterion (HSIC) to robustly capture nonlinear dependencies within a Reproducing Kernel Hilbert Space, enriching representation learning. Experimental evaluations on diverse benchmarks demonstrate the efficacy of CDSSL in improving representation quality.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 31 Jan 2025.
Builds on
17 of 34 references.
References · 17
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2020Exploring Simple Siamese Representation Learningcited by 2,866
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
- 2020Self-supervised Learning: Generative or Contrastivecited by 694
- 2020Whitening for Self-Supervised Representation Learningcited by 279
5 more
- 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
- 2021Self-Supervised Learning with Kernel Dependence Maximizationcited by 77
- 2018Unsupervised Meta-Learning For Few-Shot Image Classificationcited by 75
- 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
- 2023Pseudo-keypoint RKHS Learning for Self-supervised 6DoF Pose Estimationcited by 4
Built on by
Citing papers per year: 2025: 3.
3 papers.