mlsanityData to 7 Oct 2026

Multi-task Self-Supervised Visual Learning

arXiv 1708.07860 ↗First posted 25 Aug 2017cs.CVcited by 200

Not on the map. Why?

Abstract (arXiv)

We investigate methods for combining multiple self-supervised tasks--i.e., supervised tasks where data can be collected without manual labeling--in order to train a single visual representation. First, we provide an apples-to-apples comparison of four different self-supervised tasks using the very deep ResNet-101 architecture. We then combine tasks to jointly train a network. We also explore lasso regularization to encourage the network to factorize the information in its representation, and methods for "harmonizing" network inputs in order to learn a more unified representation. We evaluate all methods on ImageNet classification, PASCAL VOC detection, and NYU depth prediction. Our results show that deeper networks work better, and that combining tasks--even via a naive multi-head architecture--always improves performance. Our best joint network nearly matches the PASCAL performance of a model pre-trained on ImageNet classification, and matches the ImageNet network on NYU depth prediction.

Key facts. Cited by 200 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 1; 2021: 61; 2022: 56; 2023: 34; 2024: 27; 2025: 13; 2026: 7.

200 papers.

Sort:

By area:

  1. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  2. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  3. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  4. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  5. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  6. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  7. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  8. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  9. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  10. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  11. 2021Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representationscited by 96
  12. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
38 more
  1. 2021Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptationcited by 83
  2. 2021Self-Supervised Pretraining Improves Self-Supervised Pretrainingcited by 81
  3. 2022Object discovery and representation networkscited by 80
  4. 2023Independent Component Alignment for Multi-Task Learningcited by 69
  5. 2021Spatially Consistent Representation Learningcited by 64
  6. 2021CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learningcited by 64
  7. 2021Self-supervised Video Transformercited by 64
  8. 2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
  9. 2021Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworkscited by 56
  10. 2022UniCLIP: Unified Framework for Contrastive Language-Image Pre-trainingcited by 54
  11. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  12. 2021Automated Self-Supervised Learning for Graphscited by 50
  13. 2021Online Adversarial Purification based on Self-Supervisioncited by 45
  14. 2023Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspectivecited by 43
  15. 2022UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translationcited by 42
  16. 2022Robust and Efficient Medical Imaging with Self-Supervisioncited by 40
  17. 2022Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrastcited by 37
  18. 2022Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalizationcited by 35
  19. 2021Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labelscited by 32
  20. 2022Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Imagescited by 32
  21. 2021SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Datacited by 31
  22. 20213D Spatial Recognition without Spatially Labeled 3Dcited by 31
  23. 2021Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learningcited by 30
  24. 2022Can't Steal? Cont-Steal! Contrastive Stealing Attacks Against Image Encoderscited by 30
  25. 2022A Simple Data Mixing Prior for Improving Self-Supervised Learningcited by 27
  26. 2021Self-Supervised Multi-Object Tracking with Cross-Input Consistencycited by 25
  27. 2024PhilEO Bench: Evaluating Geo-Spatial Foundation Modelscited by 25
  28. 2022Universal Representations: A Unified Look at Multiple Task and Domain Learningcited by 23
  29. 2021Interpretability-Driven Sample Selection Using Self Supervised Learning For Disease Classification And Segmentationcited by 22
  30. 2022Unsupervised Visual Representation Learning by Synchronous Momentum Groupingcited by 22
  31. 2022Open-set Adversarial Defense with Clean-Adversarial Mutual Learningcited by 21
  32. 2022Extreme Masking for Learning Instance and Distributed Visual Representationscited by 21
  33. 2022Graph Neural Networks for Moleculescited by 21
  34. 2022Beyond Supervised vs. Unsupervised: Representative Benchmarking and Analysis of Image Representation Learningcited by 20
  35. 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
  36. 2024Self-supervised Learning for Electroencephalogram: A Systematic Surveycited by 19
  37. 2024Learning Graph Quantized Tokenizerscited by 19
  38. 2021Weakly supervised segmentation with cross-modality equivariant constraintscited by 18

50 most cited of 200 All 200 (JSON)