# DINE: Domain Adaptation from Single and Multiple Black-box Predictors

arXiv: https://arxiv.org/abs/2104.01539 · HTML: https://mlsanity.com/paper/2104.01539/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 89 references that are in the map.
- First posted on arXiv 4 Apr 2021.

## Builds on (most cited first, 30 of 56)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Strong-Weak Distribution Alignment for Adaptive Object Detection](https://mlsanity.com/paper/1812.04798/) (2018)
- [Interpolation Consistency Training for Semi-Supervised Learning](https://mlsanity.com/paper/1903.03825/) (2019)

## Built on by (most cited first, 30 of 51)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [Black-box Prompt Learning for Pre-trained Language Models](https://mlsanity.com/paper/2201.08531/) (2022)
- [DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices](https://mlsanity.com/paper/2303.13325/) (2023)
- [EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2403.14082/) (2024)
- [Prior Knowledge Guided Unsupervised Domain Adaptation](https://mlsanity.com/paper/2207.08877/) (2022)
- [Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors](https://mlsanity.com/paper/2205.14467/) (2022)
- [Active Test-Time Adaptation: Theoretical Analyses and An Algorithm](https://mlsanity.com/paper/2404.05094/) (2024)
- [Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?](https://mlsanity.com/paper/2305.18712/) (2023)
- [Make the U in UDA Matter: Invariant Consistency Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2309.12742/) (2023)
- [A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation](https://mlsanity.com/paper/2106.11653/) (2021)
- [Black-box Unsupervised Domain Adaptation with Bi-directional Atkinson-Shiffrin Memory](https://mlsanity.com/paper/2308.13236/) (2023)
- [RAIN: RegulArization on Input and Network for Black-Box Domain Adaptation](https://mlsanity.com/paper/2208.10531/) (2022)
- [Fast and Accurate Transferability Measurement by Evaluating Intra-class Feature Variance](https://mlsanity.com/paper/2308.05986/) (2023)
- [More is Better: Deep Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2405.00749/) (2024)
- [MDFL: Multi-domain Diffusion-driven Feature Learning](https://mlsanity.com/paper/2311.09520/) (2023)
- [Personalized Feature Translation for Expression Recognition: An Efficient Source-Free Domain Adaptation Method](https://mlsanity.com/paper/2508.09202/) (2025)
- [Leveraging Endo- and Exo-Temporal Regularization for Black-box Video Domain Adaptation](https://mlsanity.com/paper/2208.05187/) (2022)
- [Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptation](https://mlsanity.com/paper/2303.10451/) (2023)
- [Generalizing vision-language models to novel domains: A comprehensive survey](https://mlsanity.com/paper/2506.18504/) (2025)
- [Exploiting Inter-Sample Affinity for Knowability-Aware Universal Domain Adaptation](https://mlsanity.com/paper/2207.09280/) (2022)
- [Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition](https://mlsanity.com/paper/2210.05246/) (2022)
- [UFDA: Universal Federated Domain Adaptation with Practical Assumptions](https://mlsanity.com/paper/2311.15570/) (2023)
- [MAPS: A Noise-Robust Progressive Learning Approach for Source-Free Domain Adaptive Keypoint Detection](https://mlsanity.com/paper/2302.04589/) (2023)
- [CoSDA: Continual Source-Free Domain Adaptation](https://mlsanity.com/paper/2304.06627/) (2023)
- [Continual Domain Adversarial Adaptation via Double-Head Discriminators](https://mlsanity.com/paper/2402.03588/) (2024)
- [Supervision Interpolation via LossMix: Generalizing Mixup for Object Detection and Beyond](https://mlsanity.com/paper/2303.10343/) (2023)
- [AdaptGuard: Defending Against Universal Attacks for Model Adaptation](https://mlsanity.com/paper/2303.10594/) (2023)
- [Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation](https://mlsanity.com/paper/2312.06331/) (2023)
- [Cross-Domain Transfer Learning with CoRTe: Consistent and Reliable Transfer from Black-Box to Lightweight Segmentation Model](https://mlsanity.com/paper/2402.13122/) (2024)

## Abstract (arXiv)

To ease the burden of labeling, unsupervised domain adaptation (UDA) aims to transfer knowledge in previous and related labeled datasets (sources) to a new unlabeled dataset (target). Despite impressive progress, prior methods always need to access the raw source data and develop data-dependent alignment approaches to recognize the target samples in a transductive learning manner, which may raise privacy concerns from source individuals. Several recent studies resort to an alternative solution by exploiting the well-trained white-box model from the source domain, yet, it may still leak the raw data through generative adversarial learning. This paper studies a practical and interesting setting for UDA, where only black-box source models (i.e., only network predictions are available) are provided during adaptation in the target domain. To solve this problem, we propose a new two-step knowledge adaptation framework called DIstill and fine-tuNE (DINE). Taking into consideration the target data structure, DINE first distills the knowledge from the source predictor to a customized target model, then fine-tunes the distilled model to further fit the target domain. Besides, neural networks are not required to be identical across domains in DINE, even allowing effective adaptation on a low-resource device. Empirical results on three UDA scenarios (i.e., single-source, multi-source, and partial-set) confirm that DINE achieves highly competitive performance compared to state-of-the-art data-dependent approaches. Code is available at urlhttps://github.com/tim-learn/DINE/.