# Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation

arXiv: https://arxiv.org/abs/2107.06154 · HTML: https://mlsanity.com/paper/2107.06154/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 72 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs](https://mlsanity.com/paper/1803.08035/) (2018)
- [Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1705.00609/) (2017)
- [Domain Adaptation for Semantic Segmentation with Maximum Squares Loss](https://mlsanity.com/paper/1909.13589/) (2019)
- [Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations](https://mlsanity.com/paper/2003.12237/) (2020)
- [Gradually Vanishing Bridge for Adversarial Domain Adaptation](https://mlsanity.com/paper/2003.13183/) (2020)
- [Domain-Symmetric Networks for Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.04663/) (2019)
- [Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift](https://mlsanity.com/paper/1803.00830/) (2018)
- [Adversarial Dropout Regularization](https://mlsanity.com/paper/1711.01575/) (2017)

## Built on by (most cited first, 14 of 14)

- [Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation](https://mlsanity.com/paper/2205.04183/) (2022)
- [Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation](https://mlsanity.com/paper/2204.03838/) (2022)
- [BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation](https://mlsanity.com/paper/2204.02811/) (2022)
- [CoNMix for Source-free Single and Multi-target Domain Adaptation](https://mlsanity.com/paper/2211.03876/) (2022)
- [Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation](https://mlsanity.com/paper/2302.01094/) (2023)
- [Three New Validators and a Large-Scale Benchmark Ranking for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2208.07360/) (2022)
- [Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error](https://mlsanity.com/paper/2308.03003/) (2023)
- [Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classification](https://mlsanity.com/paper/2502.10694/) (2025)
- [De-Confusing Pseudo-Labels in Source-Free Domain Adaptation](https://mlsanity.com/paper/2401.01650/) (2024)
- [Polycentric Clustering and Structural Regularization for Source-free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2210.07463/) (2022)
- [Graph Harmony: Denoising and Nuclear-Norm Wasserstein Adaptation for Enhanced Domain Transfer in Graph-Structured Data](https://mlsanity.com/paper/2301.12361/) (2023)
- [Diffusion-Driven Progressive Target Manipulation for Source-Free Domain Adaptation](https://mlsanity.com/paper/2510.25279/) (2025)
- [Sequential Recommendation with Controllable Diversification: Representation Degeneration and Diversity](https://mlsanity.com/paper/2306.11986/) (2023)
- [Trust And Balance: Few Trusted Samples Pseudo-Labeling and Temperature Scaled Loss for Effective Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2409.00741/) (2024)

## Abstract (arXiv)

Due to the domain discrepancy in visual domain adaptation, the performance of source model degrades when bumping into the high data density near decision boundary in target domain. A common solution is to minimize the Shannon Entropy to push the decision boundary away from the high density area. However, entropy minimization also leads to severe reduction of prediction diversity, and unfortunately brings harm to the domain adaptation. In this paper, we investigate the prediction discriminability and diversity by studying the structure of the classification output matrix of a randomly selected data batch. We find by theoretical analysis that the prediction discriminability and diversity could be separately measured by the Frobenius-norm and rank of the batch output matrix. The nuclear-norm is an upperbound of the former, and a convex approximation of the latter. Accordingly, we propose Batch Nuclear-norm Maximization and Minimization, which performs nuclear-norm maximization on the target output matrix to enhance the target prediction ability, and nuclear-norm minimization on the source batch output matrix to increase applicability of the source domain knowledge. We further approximate the nuclear-norm by L_{1,2}-norm, and design multi-batch optimization for stable solution on large number of categories. The fast approximation method achieves O(n^2) computational complexity and better convergence property. Experiments show that our method could boost the adaptation accuracy and robustness under three typical domain adaptation scenarios. The code is available at https://github.com/cuishuhao/BNM.