# Self-supervised learning of Split Invariant Equivariant representations

arXiv: https://arxiv.org/abs/2302.10283 · HTML: https://mlsanity.com/paper/2302.10283/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 56 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Whitening for Self-Supervised Representation Learning](https://mlsanity.com/paper/2007.06346/) (2020)
- [What Should Not Be Contrastive in Contrastive Learning](https://mlsanity.com/paper/2008.05659/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Contrastive Learning of Structured World Models](https://mlsanity.com/paper/1911.12247/) (2019)
- [Efficient Self-supervised Vision Transformers for Representation Learning](https://mlsanity.com/paper/2106.09785/) (2021)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [VICRegL: Self-Supervised Learning of Local Visual Features](https://mlsanity.com/paper/2210.01571/) (2022)
- [On the duality between contrastive and non-contrastive self-supervised learning](https://mlsanity.com/paper/2206.02574/) (2022)

## Built on by (most cited first, 30 of 44)

- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Learning and Leveraging World Models in Visual Representation Learning](https://mlsanity.com/paper/2403.00504/) (2024)
- [MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features](https://mlsanity.com/paper/2307.12698/) (2023)
- [Self-Supervised Learning with Lie Symmetries for Partial Differential Equations](https://mlsanity.com/paper/2307.05432/) (2023)
- [Latent Space Symmetry Discovery](https://mlsanity.com/paper/2310.00105/) (2023)
- [Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning](https://mlsanity.com/paper/2306.13924/) (2023)
- [Learning Infinitesimal Generators of Continuous Symmetries from Data](https://mlsanity.com/paper/2410.21853/) (2024)
- [EquiAV: Leveraging Equivariance for Audio-Visual Contrastive Learning](https://mlsanity.com/paper/2403.09502/) (2024)
- [Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization](https://mlsanity.com/paper/2403.14973/) (2024)
- [Collapse-Proof Non-Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2410.04959/) (2024)
- [Understanding the Role of Equivariance in Self-supervised Learning](https://mlsanity.com/paper/2411.06508/) (2024)
- [seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models](https://mlsanity.com/paper/2505.03176/) (2025)
- [Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks](https://mlsanity.com/paper/2505.06224/) (2025)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [In-Context Symmetries: Self-Supervised Learning through Contextual World Models](https://mlsanity.com/paper/2405.18193/) (2024)
- [Self-Supervised Learning for Place Representation Generalization across Appearance Changes](https://mlsanity.com/paper/2303.02370/) (2023)
- [Center-Based Relaxed Learning Against Membership Inference Attacks](https://mlsanity.com/paper/2404.17674/) (2024)
- [Incorporating Arbitrary Matrix Group Equivariance into KANs](https://mlsanity.com/paper/2410.00435/) (2024)
- [Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection](https://mlsanity.com/paper/2404.11737/) (2024)
- [PESTO: Real-Time Pitch Estimation with Self-supervised Transposition-equivariant Objective](https://mlsanity.com/paper/2508.01488/) (2025)
- [Augmentation-aware Self-supervised Learning with Conditioned Projector](https://mlsanity.com/paper/2306.06082/) (2023)
- [Capsule Network Projectors are Equivariant and Invariant Learners](https://mlsanity.com/paper/2405.14386/) (2024)
- [Equivariant Representation Learning for Augmentation-based Self-Supervised Learning via Image Reconstruction](https://mlsanity.com/paper/2412.03314/) (2024)
- [MatrixNet: Learning over symmetry groups using learned group representations](https://mlsanity.com/paper/2501.09571/) (2025)
- [Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions](https://mlsanity.com/paper/2510.21706/) (2025)
- [Learning Temporally Equivariance for Degenerative Disease Progression in OCT by Predicting Future Representations](https://mlsanity.com/paper/2405.09404/) (2024)
- [Contrastive Learning Via Equivariant Representation](https://mlsanity.com/paper/2406.00262/) (2024)
- [Point-PNG: Conditional Pseudo-Negatives Generation for Point Cloud Pre-Training](https://mlsanity.com/paper/2409.15832/) (2024)
- [Self-Supervised Learning Based on Transformed Image Reconstruction for Equivariance-Coherent Feature Representation](https://mlsanity.com/paper/2503.18753/) (2025)
- [OWT: A Foundational Organ-Wise Tokenization Framework for Medical Imaging](https://mlsanity.com/paper/2505.04899/) (2025)

## Abstract (arXiv)

Recent progress has been made towards learning invariant or equivariant representations with self-supervised learning. While invariant methods are evaluated on large scale datasets, equivariant ones are evaluated in smaller, more controlled, settings. We aim at bridging the gap between the two in order to learn more diverse representations that are suitable for a wide range of tasks. We start by introducing a dataset called 3DIEBench, consisting of renderings from 3D models over 55 classes and more than 2.5 million images where we have full control on the transformations applied to the objects. We further introduce a predictor architecture based on hypernetworks to learn equivariant representations with no possible collapse to invariance. We introduce SIE (Split Invariant-Equivariant) which combines the hypernetwork-based predictor with representations split in two parts, one invariant, the other equivariant, to learn richer representations. We demonstrate significant performance gains over existing methods on equivariance related tasks from both a qualitative and quantitative point of view. We further analyze our introduced predictor and show how it steers the learned latent space. We hope that both our introduced dataset and approach will enable learning richer representations without supervision in more complex scenarios. Code and data are available at https://github.com/facebookresearch/SIE.