mlsanityData to 7 Oct 2026

Self-Supervised Learning with a Multi-Task Latent Space Objective

arXiv 2602.05845 ↗First posted 5 Feb 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a multi-task formulation of self-predictive Siamese SSL in which each spatial transformation defines a distinct latent-space alignment task, solved by a dedicated predictor over a shared encoder. This perspective directly explains a long-standing failure of multi-crop training in self-predictive methods such as BYOL, SimSiam, and MoCo v3: a shared predictor is forced to solve heterogeneous alignment tasks simultaneously, leading to unstable optimization. Assigning one predictor per view type resolves this interference, unlocking linear evaluation gains of 3.8-4% across frameworks. This perspective also suggests a principled way to enrich pre-training by introducing additional spatial transformations as complementary tasks. We demonstrate this by introducing asymmetric cutout views, in which a masked online view is aligned with a complete target, forming a semantic inpainting objective. The resulting framework is stable, backbone-agnostic, and consistently improves the performance of ResNet and ViT models on ImageNet and COCO.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 67 of its 75 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

67 of 75 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  12. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  13. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  14. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  15. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  16. 2025DINOv3cited by 1,297
  17. 2017Random Erasing Data Augmentationcited by 1,235
  18. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  19. 2016Colorful Image Colorizationcited by 1,113
  20. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  21. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  22. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  23. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  24. 2016Deep Networks with Stochastic Depthcited by 973
  25. 2017Large Batch Training of Convolutional Networkscited by 529
  26. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  27. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  28. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  29. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  30. 2020Whitening for Self-Supervised Representation Learningcited by 278
  31. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
  32. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  33. 2017Multi-task Self-Supervised Visual Learningcited by 200
  34. 2022Multimodal Masked Autoencoders Learn Transferable Representationscited by 107
  35. 2021Region Similarity Representation Learningcited by 96
  36. 2020MS2L: Multi-Task Self-Supervised Learning for Skeleton Based Action Recognitioncited by 92
  37. 2019Unsupervised Multi-Task Feature Learning on Point Cloudscited by 89
  38. 2022Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?cited by 67

50 most cited of 67 All 67 (JSON)

Built on by

Not cited yet.