Homomorphic Self-Supervised Learning
Not on the map. Why?
Abstract (arXiv)
In this work, we observe that many existing self-supervised learning algorithms can be both unified and generalized when seen through the lens of equivariant representations. Specifically, we introduce a general framework we call Homomorphic Self-Supervised Learning, and theoretically show how it may subsume the use of input-augmentations provided an augmentation-homomorphic feature extractor. We validate this theory experimentally for simple augmentations, demonstrate how the framework fails when representational structure is removed, and further empirically explore how the parameters of this framework relate to those of traditional augmentation-based self-supervised learning. We conclude with a discussion of the potential benefits afforded by this new perspective on self-supervised learning.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 15 Nov 2022.
Builds on
44 of 58 references.
References · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2020Exploring Simple Siamese Representation Learningcited by 2,853
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2019Contrastive Multiview Codingcited by 1,297
- 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2020What Makes for Good Views for Contrastive Learning?cited by 843
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
32 more
- 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
- 2017Large Batch Training of Convolutional Networkscited by 529
- 2019General E(2)-Equivariant Steerable CNNscited by 441
- 2019Making Convolutional Networks Shift-Invariant Againcited by 390
- 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
- 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
- 2016Steerable CNNscited by 298
- 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
- 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
- 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
- 2020What Should Not Be Contrastive in Contrastive Learningcited by 224
- 2020Learning Robust Representations via Multi-View Information Bottleneckcited by 189
- 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
- 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
- 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
- 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
- 2017Dataset Augmentation in Feature Spacecited by 124
- 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
- 2019Scale-Equivariant Steerable Networkscited by 110
- 2022Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlapcited by 105
- 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
- 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
- 2022Adversarial Masking for Self-Supervised Learningcited by 78
- 2020Understanding Self-supervised Learning with Dual Deep Networkscited by 62
- 2021The Power of Contrast for Feature Learning: A Theoretical Analysiscited by 53
- 2020Variational Autoencoder with Learned Latent Structurecited by 41
- 2021Residual Relaxation for Multi-view Representation Learningcited by 30
- 2022Conditional Contrastive Learning with Kernelcited by 22
- 2022Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitionscited by 18
- 2021Challenges of Adversarial Image Augmentationscited by 4
- 2018Non-Parametric Transformation Networkscited by 2
- 2021FILTRA: Rethinking Steerable CNN by Filter Transformcited by 1
Built on by
Citing papers per year: 2023: 2; 2024: 0; 2025: 1.
3 papers.