# RAIN: RegulArization on Input and Network for Black-Box Domain Adaptation

arXiv: https://arxiv.org/abs/2208.10531 · HTML: https://mlsanity.com/paper/2208.10531/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 22 Aug 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space](https://mlsanity.com/paper/2103.06030/) (2021)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering](https://mlsanity.com/paper/2003.08607/) (2020)
- [Dual Mixup Regularized Learning for Adversarial Domain Adaptation](https://mlsanity.com/paper/2007.03141/) (2020)
- [Domain Impression: A Source Data Free Domain Adaptation Method](https://mlsanity.com/paper/2102.09003/) (2021)
- [Unsupervised Multi-source Domain Adaptation Without Access to Source Data](https://mlsanity.com/paper/2104.01845/) (2021)
- [DINE: Domain Adaptation from Single and Multiple Black-box Predictors](https://mlsanity.com/paper/2104.01539/) (2021)
- [Unsupervised Domain Adaptation of Black-Box Source Models](https://mlsanity.com/paper/2101.02839/) (2021)
- [Visualizing Adapted Knowledge in Domain Transfer](https://mlsanity.com/paper/2104.10602/) (2021)

## Built on by (most cited first, 9 of 9)

- [Automatic News Generation and Fact-Checking System Based on Language Processing](https://mlsanity.com/paper/2405.10492/) (2024)
- [Optimizing Automated Picking Systems in Warehouse Robots Using Machine Learning](https://mlsanity.com/paper/2408.16633/) (2024)
- [Text Sentiment Analysis and Classification Based on Bidirectional Gated Recurrent Units (GRUs) Model](https://mlsanity.com/paper/2404.17123/) (2024)
- [Rank and Align: Towards Effective Source-free Graph Domain Adaptation](https://mlsanity.com/paper/2408.12185/) (2024)
- [A Novel Cross-Perturbation for Single Domain Generalization](https://mlsanity.com/paper/2308.00918/) (2023)
- [Incremental Pseudo-Labeling for Black-Box Unsupervised Domain Adaptation](https://mlsanity.com/paper/2405.16437/) (2024)
- [FIESTA: Fourier-Based Semantic Augmentation with Uncertainty Guidance for Enhanced Domain Generalizability in Medical Image Segmentation](https://mlsanity.com/paper/2406.14308/) (2024)
- [High-order Neighborhoods Know More: HyperGraph Learning Meets Source-free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2405.06916/) (2024)
- [Adaptive Dual-Teacher Distillation with Subnetwork Rectification for Bridging Semantic Gaps in Black-Box Domain Adaptation](https://mlsanity.com/paper/2603.22908/) (2026)

## Abstract (arXiv)

Source-Free domain adaptation transits the source-trained model towards target domain without exposing the source data, trying to dispel these concerns about data privacy and security. However, this paradigm is still at risk of data leakage due to adversarial attacks on the source model. Hence, the Black-Box setting only allows to use the outputs of source model, but still suffers from overfitting on the source domain more severely due to source model's unseen weights. In this paper, we propose a novel approach named RAIN (RegulArization on Input and Network) for Black-Box domain adaptation from both input-level and network-level regularization. For the input-level, we design a new data augmentation technique as Phase MixUp, which highlights task-relevant objects in the interpolations, thus enhancing input-level regularization and class consistency for target models. For network-level, we develop a Subnetwork Distillation mechanism to transfer knowledge from the target subnetwork to the full target network via knowledge distillation, which thus alleviates overfitting on the source domain by learning diverse target representations. Extensive experiments show that our method achieves state-of-the-art performance on several cross-domain benchmarks under both single- and multi-source black-box domain adaptation.