# Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery

arXiv: https://arxiv.org/abs/1711.09082 · HTML: https://mlsanity.com/paper/1711.09082/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2017.

## Built on by (most cited first, 30 of 56)

- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [Scaling Laws of Synthetic Images for Model Training ... for Now](https://mlsanity.com/paper/2312.04567/) (2023)
- [Generative Models as a Data Source for Multiview Representation Learning](https://mlsanity.com/paper/2106.05258/) (2021)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Monocular Depth Estimation using Diffusion Models](https://mlsanity.com/paper/2302.14816/) (2023)
- [What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?](https://mlsanity.com/paper/2106.05961/) (2021)
- [Automated Self-Supervised Learning for Graphs](https://mlsanity.com/paper/2106.05470/) (2021)
- [Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision](https://mlsanity.com/paper/2104.01257/) (2021)
- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [TIPS: Text-Image Pretraining with Spatial awareness](https://mlsanity.com/paper/2410.16512/) (2024)
- [Single Image Depth Estimation: An Overview](https://mlsanity.com/paper/2104.06456/) (2021)
- [Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalization](https://mlsanity.com/paper/2210.02016/) (2022)
- [3D Spatial Recognition without Spatially Labeled 3D](https://mlsanity.com/paper/2105.06461/) (2021)
- [Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data](https://mlsanity.com/paper/2112.00054/) (2021)
- [Do Generated Data Always Help Contrastive Learning?](https://mlsanity.com/paper/2403.12448/) (2024)
- [Semi-supervised Multi-task Learning for Semantics and Depth](https://mlsanity.com/paper/2110.07197/) (2021)
- [Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation](https://mlsanity.com/paper/2207.13247/) (2022)
- [Fully Self-Supervised Depth Estimation from Defocus Clue](https://mlsanity.com/paper/2303.10752/) (2023)
- [Generative Modeling for Multi-task Visual Learning](https://mlsanity.com/paper/2106.13409/) (2021)
- [RIO: Rotation-equivariance supervised learning of robust inertial odometry](https://mlsanity.com/paper/2111.11676/) (2021)
- [DeNVeR: Deformable Neural Vessel Representations for Unsupervised Video Vessel Segmentation](https://mlsanity.com/paper/2406.01591/) (2024)
- [CoCon: Cooperative-Contrastive Learning](https://mlsanity.com/paper/2104.14764/) (2021)
- [Multi-Task Learning of Object State Changes from Uncurated Videos](https://mlsanity.com/paper/2211.13500/) (2022)
- [SSLayout360: Semi-Supervised Indoor Layout Estimation from 360-Degree Panorama](https://mlsanity.com/paper/2103.13696/) (2021)
- [ScatSimCLR: self-supervised contrastive learning with pretext task regularization for small-scale datasets](https://mlsanity.com/paper/2108.13939/) (2021)

## Abstract (arXiv)

In human learning, it is common to use multiple sources of information jointly. However, most existing feature learning approaches learn from only a single task. In this paper, we propose a novel multi-task deep network to learn generalizable high-level visual representations. Since multi-task learning requires annotations for multiple properties of the same training instance, we look to synthetic images to train our network. To overcome the domain difference between real and synthetic data, we employ an unsupervised feature space domain adaptation method based on adversarial learning. Given an input synthetic RGB image, our network simultaneously predicts its surface normal, depth, and instance contour, while also minimizing the feature space domain differences between real and synthetic data. Through extensive experiments, we demonstrate that our network learns more transferable representations compared to single-task baselines. Our learned representation produces state-of-the-art transfer learning results on PASCAL VOC 2007 classification and 2012 detection.