# Sequential Domain Adaptation by Synthesizing Distributionally Robust Experts

arXiv: https://arxiv.org/abs/2106.00322 · HTML: https://mlsanity.com/paper/2106.00322/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 74 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Simultaneous Deep Transfer Across Domains and Tasks](https://mlsanity.com/paper/1510.02192/) (2015)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1607.03516/) (2016)
- [Regularized Learning for Domain Adaptation under Label Shifts](https://mlsanity.com/paper/1903.09734/) (2019)
- [Few-Shot Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.02536/) (2017)
- [Adaptive Gradient Descent without Descent](https://mlsanity.com/paper/1910.09529/) (2019)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1806.00258/) (2018)
- [Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionality](https://mlsanity.com/paper/2009.04382/) (2020)
- [Optimal learning with Bernstein Online Aggregation](https://mlsanity.com/paper/1404.1356/) (2014)
- [Recovering Best Statistical Guarantees via the Empirical Divergence-based Distributionally Robust Optimization](https://mlsanity.com/paper/1605.09349/) (2016)
- [Robust Hypothesis Testing Using Wasserstein Uncertainty Sets](https://mlsanity.com/paper/1805.10611/) (2018)
- [Adversarial Weighting for Domain Adaptation in Regression](https://mlsanity.com/paper/2006.08251/) (2020)
- [Domain Adaptation by Mixture of Alignments of Second- or Higher-Order Scatter Tensors](https://mlsanity.com/paper/1611.08195/) (2016)
- [Wasserstein Distributionally Robust Kalman Filtering](https://mlsanity.com/paper/1809.08830/) (2018)
- [Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation](https://mlsanity.com/paper/1910.10583/) (2019)
- [Semi-Supervised Domain Adaptation with Non-Parametric Copulas](https://mlsanity.com/paper/1301.0142/) (2013)
- [Calculating Optimistic Likelihoods Using (Geodesically) Convex Optimization](https://mlsanity.com/paper/1910.07817/) (2019)
- [Robust Bayesian Classification Using an Optimistic Score Ratio](https://mlsanity.com/paper/2007.04458/) (2020)

## Built on by (most cited first, 14 of 14)

- [Counterfactual Plans under Distributional Ambiguity](https://mlsanity.com/paper/2201.12487/) (2022)
- [Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data](https://mlsanity.com/paper/2106.07636/) (2021)
- [Robust Bayesian Recourse](https://mlsanity.com/paper/2206.10833/) (2022)
- [Distributionally Robust Recourse Action](https://mlsanity.com/paper/2302.11211/) (2023)
- [Distributionally Robust Fair Principal Components via Geodesic Descents](https://mlsanity.com/paper/2202.03071/) (2022)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [A Unified Data Representation Learning for Non-parametric Two-sample Testing](https://mlsanity.com/paper/2412.00613/) (2024)
- [Universal generalization guarantees for Wasserstein distributionally robust models](https://mlsanity.com/paper/2402.11981/) (2024)
- [Contextual Optimization under Covariate Shift: A Robust Approach by Intersecting Wasserstein Balls](https://mlsanity.com/paper/2406.02426/) (2024)
- [Wasserstein Distributionally Robust Optimization with Wasserstein Barycenters](https://mlsanity.com/paper/2203.12136/) (2022)
- [Distributionally Robust Domain Adaptation](https://mlsanity.com/paper/2210.16894/) (2022)
- [A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set](https://mlsanity.com/paper/2405.20124/) (2024)
- [Distributionally and Adversarially Robust Logistic Regression via Intersecting Wasserstein Balls](https://mlsanity.com/paper/2407.13625/) (2024)
- [Wasserstein Distributionally Robust Risk-Sensitive Estimation via Conditional Value-at-Risk](https://mlsanity.com/paper/2604.18546/) (2026)

## Abstract (arXiv)

Least squares estimators, when trained on a few target domain samples, may predict poorly. Supervised domain adaptation aims to improve the predictive accuracy by exploiting additional labeled training samples from a source distribution that is close to the target distribution. Given available data, we investigate novel strategies to synthesize a family of least squares estimator experts that are robust with regard to moment conditions. When these moment conditions are specified using Kullback-Leibler or Wasserstein-type divergences, we can find the robust estimators efficiently using convex optimization. We use the Bernstein online aggregation algorithm on the proposed family of robust experts to generate predictions for the sequential stream of target test samples. Numerical experiments on real data show that the robust strategies may outperform non-robust interpolations of the empirical least squares estimators.