mlsanityData to 14 Oct 2026

Variance Matters: Improving Domain Adaptation via Stratified Sampling

arXiv 2512.05226 ↗First posted 4 Dec 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Domain shift remains a key challenge in deploying machine learning models to the real world. Unsupervised domain adaptation (UDA) aims to address this by minimising domain discrepancy during training, but the discrepancy estimates suffer from high variance in stochastic settings, which can stifle the theoretical benefits of the method. This paper proposes Variance-Reduced Domain Adaptation via Stratified Sampling (VaRDASS), the first specialised stochastic variance reduction technique for UDA. We consider two specific discrepancy measures -- correlation alignment and the maximum mean discrepancy (MMD) -- and derive ad hoc stratification objectives for these terms. We then present expected and worst-case error bounds, and prove that our proposed objective for the MMD is theoretically optimal (i.e., minimises the variance) under certain assumptions. Finally, a practical k-means style optimisation algorithm is introduced and analysed. Experiments on four domain shift datasets demonstrate improved discrepancy estimation accuracy and target domain performance.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 35 of its 66 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

35 of 66 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  4. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
  5. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  6. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  7. 2020In Search of Lost Domain Generalizationcited by 1,054
  8. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 600
  9. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
  10. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  11. 2018Characterizing and Avoiding Negative Transfercited by 229
  12. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
23 more
  1. 2020Learning Deep Kernels for Non-Parametric Two-Sample Testscited by 130
  2. 2015Online Batch Selection for Faster Training of Neural Networkscited by 122
  3. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  4. 2014Stochastic Optimization with Importance Samplingcited by 91
  5. 2022A Closer Look at Smoothness in Domain Adversarial Trainingcited by 85
  6. 2015Variance Reduction in SGD by Distributed Importance Samplingcited by 79
  7. 2020Variance-Reduced Methods for Machine Learningcited by 79
  8. 2021Adaptive Methods for Real-World Domain Generalizationcited by 66
  9. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  10. 2023Free Lunch for Domain Adversarial Training: Environment Label Smoothingcited by 33
  11. 2017Biased Importance Sampling for Deep Neural Network Trainingcited by 31
  12. 2015A Test of Relative Similarity For Model Selection in Generative Modelscited by 25
  13. 2014Accelerating Minibatch Stochastic Gradient Descent using Stratified Samplingcited by 21
  14. 2018Active Mini-Batch Sampling using Repulsive Point Processescited by 20
  15. 2014Kernel Mean Shrinkage Estimatorscited by 16
  16. 2021Variance Reduced Training with Stratified Sampling for Forecasting Modelscited by 14
  17. 2021Determinantal point processes based on orthogonal polynomials for sampling minibatches in SGDcited by 12
  18. 2019Accelerating Minibatch Stochastic Gradient Descent using Typicality Samplingcited by 11
  19. 2019Unbiased estimators for the variance of MMD estimatorscited by 10
  20. 2018Accelerating Stochastic Gradient Descent Using Antithetic Samplingcited by 4
  21. 2013Testing Hypotheses by Regularized Maximum Mean Discrepancycited by 3
  22. 2014Scalable Kernel Clustering: Approximate Kernel k-meanscited by 2
  23. 2025Exploring Variance Reduction in Importance Sampling for Efficient DNN Trainingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Order Matters: Improving Domain Adaptation by Reordering Datacited by 2
  2. 2026Variance-reduced Domain Adaptation using Paired Samplingcited by 0
  3. 2026Online Variance Reduction for Domain Adaptation on Streaming Datacited by 0