# Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations

arXiv: https://arxiv.org/abs/2003.12237 · HTML: https://mlsanity.com/paper/2003.12237/

## Key facts

- Cited by 133 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2020.

## Built on by (most cited first, 30 of 133)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation](https://mlsanity.com/paper/2110.04202/) (2021)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation](https://mlsanity.com/paper/2106.15326/) (2021)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation](https://mlsanity.com/paper/2205.04183/) (2022)
- [Cross-domain Contrastive Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.05528/) (2021)
- [Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.04151/) (2021)
- [Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation](https://mlsanity.com/paper/2204.03838/) (2022)
- [Transferable Semantic Augmentation for Domain Adaptation](https://mlsanity.com/paper/2103.12562/) (2021)
- [MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.13575/) (2021)
- [Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective](https://mlsanity.com/paper/2303.13434/) (2023)
- [Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning](https://mlsanity.com/paper/2211.06612/) (2022)
- [DINE: Domain Adaptation from Single and Multiple Black-box Predictors](https://mlsanity.com/paper/2104.01539/) (2021)
- [When Source-Free Domain Adaptation Meets Learning with Noisy Labels](https://mlsanity.com/paper/2301.13381/) (2023)
- [Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation](https://mlsanity.com/paper/2210.04524/) (2022)
- [BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation](https://mlsanity.com/paper/2204.02811/) (2022)
- [Task-specific Inconsistency Alignment for Domain Adaptive Object Detection](https://mlsanity.com/paper/2203.15345/) (2022)
- [ProxyMix: Proxy-based Mixup Training with Label Refinery for Source-Free Domain Adaptation](https://mlsanity.com/paper/2205.14566/) (2022)
- [ToAlign: Task-oriented Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.10812/) (2021)
- [Deliberated Domain Bridging for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2209.07695/) (2022)
- [Transformer-Based Source-Free Domain Adaptation](https://mlsanity.com/paper/2105.14138/) (2021)
- [Generation, augmentation, and alignment: A pseudo-source domain based method for source-free domain adaptation](https://mlsanity.com/paper/2109.04015/) (2021)
- [GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation](https://mlsanity.com/paper/2303.10056/) (2023)
- [Unsupervised Domain Adaptation: A Reality Check](https://mlsanity.com/paper/2111.15672/) (2021)
- [Generalized Domain Conditioned Adaptation Network](https://mlsanity.com/paper/2103.12339/) (2021)
- [CoNMix for Source-free Single and Multi-target Domain Adaptation](https://mlsanity.com/paper/2211.03876/) (2022)
- [Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation](https://mlsanity.com/paper/2302.01094/) (2023)

## Abstract (arXiv)

The learning of the deep networks largely relies on the data with human-annotated labels. In some label insufficient situations, the performance degrades on the decision boundary with high data density. A common solution is to directly minimize the Shannon Entropy, but the side effect caused by entropy minimization, i.e., reduction of the prediction diversity, is mostly ignored. To address this issue, we reinvestigate the structure of classification output matrix of a randomly selected data batch. We find by theoretical analysis that the prediction discriminability and diversity could be separately measured by the Frobenius-norm and rank of the batch output matrix. Besides, the nuclear-norm is an upperbound of the Frobenius-norm, and a convex approximation of the matrix rank. Accordingly, to improve both discriminability and diversity, we propose Batch Nuclear-norm Maximization (BNM) on the output matrix. BNM could boost the learning under typical label insufficient learning scenarios, such as semi-supervised learning, domain adaptation and open domain recognition. On these tasks, extensive experimental results show that BNM outperforms competitors and works well with existing well-known methods. The code is available at https://github.com/cuishuhao/BNM.