# Online Variance Reduction for Domain Adaptation on Streaming Data

arXiv: https://arxiv.org/abs/2607.20374 · HTML: https://mlsanity.com/paper/2607.20374/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Adaptive Risk Minimization: Learning to Adapt to Domain Shift](https://mlsanity.com/paper/2007.02931/) (2020)
- [A survey on domain adaptation theory: learning bounds and theoretical guarantees](https://mlsanity.com/paper/2004.11829/) (2020)
- [A Closer Look at Smoothness in Domain Adversarial Training](https://mlsanity.com/paper/2206.08213/) (2022)
- [Adaptive Methods for Real-World Domain Generalization](https://mlsanity.com/paper/2103.15796/) (2021)
- [Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases](https://mlsanity.com/paper/2303.05470/) (2023)
- [Free Lunch for Domain Adversarial Training: Environment Label Smoothing](https://mlsanity.com/paper/2302.00194/) (2023)
- [Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling](https://mlsanity.com/paper/1405.3080/) (2014)
- [Active Mini-Batch Sampling using Repulsive Point Processes](https://mlsanity.com/paper/1804.02772/) (2018)
- [DOS: Diverse Outlier Sampling for Out-of-Distribution Detection](https://mlsanity.com/paper/2306.02031/) (2023)
- [Variance Reduced Training with Stratified Sampling for Forecasting Models](https://mlsanity.com/paper/2103.02062/) (2021)
- [Determinantal point processes based on orthogonal polynomials for sampling minibatches in SGD](https://mlsanity.com/paper/2112.06007/) (2021)
- [Accelerating Stochastic Gradient Descent Using Antithetic Sampling](https://mlsanity.com/paper/1810.03124/) (2018)
- [Variance Matters: Improving Domain Adaptation via Stratified Sampling](https://mlsanity.com/paper/2512.05226/) (2025)
- [A Fast and Effective Method for Euclidean Anticlustering: The Assignment-Based-Anticlustering Algorithm](https://mlsanity.com/paper/2601.06351/) (2026)
- [Diversity Is All You Need for Contrastive Learning: Spectral Bounds on Gradient Magnitudes](https://mlsanity.com/paper/2510.05767/) (2025)
- [Order Matters: Improving Domain Adaptation by Reordering Data](https://mlsanity.com/paper/2605.05084/) (2026)

## Abstract (arXiv)

This paper studies the problem of stochastic variance reduction (SVR) for the maximum mean discrepancy (MMD) and correlation alignment (CORAL) loss functions. Although various offline SVR algorithms for these losses have been proposed, these are incompatible with online, distributed, or incremental learning settings. This paper presents Adaptive vaRiance Reduction via Online reWeighting (ARROW), the first online SVR algorithm for the MMD and CORAL for streamed data. The method maintains moving average references of the alignment statistics, and adaptively reweights incoming minibatches so that the minibatch and reference statistics are aligned. Further, we propose a relaxed reweighting scheme so that the ensuing weight-optimisation problem is tractable. In experiments and simulations, we show that ARROW performs competitively with offline algorithms in terms of runtime, degree of variance reduction achieved, and target domain accuracy.