# Improving out-of-distribution generalization via multi-task self-supervised pretraining

arXiv: https://arxiv.org/abs/2003.13525 · HTML: https://mlsanity.com/paper/2003.13525/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 26 of 26)

- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization](https://mlsanity.com/paper/2110.10832/) (2021)
- [MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaption](https://mlsanity.com/paper/2103.16201/) (2021)
- [3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection](https://mlsanity.com/paper/2112.04764/) (2021)
- [Federated Learning with Domain Generalization](https://mlsanity.com/paper/2111.10487/) (2021)
- [Anomaly Detection under Distribution Shift](https://mlsanity.com/paper/2303.13845/) (2023)
- [Federated Domain Generalization: A Survey](https://mlsanity.com/paper/2306.01334/) (2023)
- [MADG: Margin-based Adversarial Learning for Domain Generalization](https://mlsanity.com/paper/2311.08503/) (2023)
- [Pareto Navigation Gradient Descent: a First-Order Algorithm for Optimization in Pareto Set](https://mlsanity.com/paper/2110.08713/) (2021)
- [Back-to-Bones: Rediscovering the Role of Backbones in Domain Generalization](https://mlsanity.com/paper/2209.01121/) (2022)
- [ZooD: Exploiting Model Zoo for Out-of-Distribution Generalization](https://mlsanity.com/paper/2210.09236/) (2022)
- [TFS-ViT: Token-Level Feature Stylization for Domain Generalization](https://mlsanity.com/paper/2303.15698/) (2023)
- [Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation](https://mlsanity.com/paper/2107.01079/) (2021)
- [TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision](https://mlsanity.com/paper/2205.08731/) (2022)
- [POEM: Polarization of Embeddings for Domain-Invariant Representations](https://mlsanity.com/paper/2305.13046/) (2023)
- [FDS: Feedback-guided Domain Synthesis with Multi-Source Conditional Diffusion Models for Domain Generalization](https://mlsanity.com/paper/2407.03588/) (2024)
- [3D Adversarial Augmentations for Robust Out-of-Domain Predictions](https://mlsanity.com/paper/2308.15479/) (2023)
- [Domain Generalization without Excess Empirical Risk](https://mlsanity.com/paper/2308.15856/) (2023)
- [MINR: Implicit Neural Representations with Masked Image Modelling](https://mlsanity.com/paper/2507.22404/) (2025)
- [Rapidly deploying on-device eye tracking by distilling visual foundation models](https://mlsanity.com/paper/2604.02509/) (2026)
- [WSSL: Weighted Self-supervised Learning Framework For Image-inpainting](https://mlsanity.com/paper/2211.13856/) (2022)
- [Global Layers: Non-IID Tabular Federated Learning](https://mlsanity.com/paper/2305.19290/) (2023)
- [Weight Averaging Improves Knowledge Distillation under Domain Shift](https://mlsanity.com/paper/2309.11446/) (2023)
- [Structure-Aware Stylized Image Synthesis for Robust Medical Image Segmentation](https://mlsanity.com/paper/2412.04296/) (2024)
- [Resilience of Vision Transformers for Domain Generalisation in the Presence of Out-of-Distribution Noisy Images](https://mlsanity.com/paper/2504.04225/) (2025)
- [RxnCLF: Contrastive Transformation-Aware Reaction Foundation Model for Improved Reactivity Prediction](https://mlsanity.com/paper/2608.06259/) (2026)

## Abstract (arXiv)

Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained using self-supervised learning are comparable to, or better than, supervised learning for domain generalization in computer vision. We introduce a new self-supervised pretext task of predicting responses to Gabor filter banks and demonstrate that multi-task learning of compatible pretext tasks improves domain generalization performance as compared to training individual tasks alone. Features learnt through self-supervision obtain better generalization to unseen domains when compared to their supervised counterpart when there is a larger domain shift between training and test distributions and even show better localization ability for objects of interest. Self-supervised feature representations can also be combined with other domain generalization methods to further boost performance.