mlsanityData to 14 Oct 2026

Soft Equivariance Regularization for Invariant Self-Supervised Learning

arXiv 2603.06693 ↗First posted 4 Mar 2026cs.CV cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) typically learns representations invariant to semantic-preserving augmentations. While effective for recognition, enforcing strong invariance can suppress transformation-dependent structure that is useful for robustness to geometric perturbations and spatially sensitive transfer. A growing body of work, therefore, augments invariance-based SSL with equivariance objectives, but these objectives are often imposed on the same final representation. We empirically observe a trade-off in this coupled setting: pushing equivariance regularization toward deeper layers improves equivariance scores but degrades ImageNet-1k linear evaluation, motivating a layer-decoupled design. Motivated by this trade-off, we propose Soft Equivariance Regularization (SER), a plug-in regularizer that decouples where invariance and equivariance are enforced: we keep the base SSL objective unchanged on the final embedding, while softly encouraging equivariance on an intermediate spatial token map via analytically specified group actions ρg applied directly in feature space. SER learns/predicts no per-sample transformation codes/labels, requires no auxiliary transformation-prediction head, and adds only 1.008x training FLOPs. On ImageNet-1k ViT-S/16 pretraining, SER improves MoCo-v3 by +0.84 Top-1 in linear evaluation under a strictly matched 2-view setting and consistently improves DINO and Barlow Twins; under matched view counts, SER achieves the best ImageNet-1k linear-eval Top-1 among the compared invariance+equivariance add-ons. SER further improves ImageNet-C/P by +1.11/+1.22 Top-1 and frozen-backbone COCO detection by +1.7 mAP. Finally, applying the same layer-decoupling recipe to existing invariance+equivariance baselinesimproves their accuracy, suggesting layer decoupling as a general design principle for combining invariance and equivariance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 40 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

36 of 40 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
24 more
  1. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  2. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  3. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  6. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  7. 2016Group Equivariant Convolutional Networkscited by 1,165
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  10. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  11. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  12. 2016Steerable CNNscited by 305
  13. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 183
  14. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  15. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 75
  16. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 72
  17. 2021Equivariant Contrastive Learningcited by 66
  18. 2022Learning Symmetric Embeddings for Equivariant World Modelscited by 54
  19. 2021Improving Transferability of Representations via Augmentation-Aware Self-Supervisioncited by 49
  20. 2023Self-supervised learning of Split Invariant Equivariant representationscited by 44
  21. 2023Structuring Representation Geometry with Rotationally Equivariant Contrastive Learningcited by 25
  22. 2022EquiMod: An Equivariance Module to Improve Self-Supervised Learningcited by 21
  23. 2022Equivariant Representation Learning via Class-Pose Decompositioncited by 17
  24. 2023Regularizing Towards Soft Equivariance Under Mixed Symmetriescited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learningcited by 1