mlsanityData to 7 Oct 2026

Maximizing Information in Domain-Invariant Representation Improves Transfer Learning

arXiv 2306.00262 ↗First posted 1 Jun 2023cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose MaxDIRep, a domain adaptation method that improves the decomposition of data representations into domain-independent and domain-dependent components. Existing methods, such as Domain-Separation Networks (DSN), use a weak orthogonality constraint between these components, which can lead to label-relevant features being partially encoded in the domain-dependent representation (DDRep) rather than the domain-independent representation (DIRep). As a result, information crucial for target-domain classification may be missing from the DIRep. MaxDIRep addresses this issue by applying a Kullback-Leibler (KL) divergence constraint to minimize the information content of the DDRep, thereby encouraging the DIRep to retain features that are both domain-invariant and predictive of target labels. Through geometric analysis and an ablation study on synthetic datasets, we show why DSN's weaker constraint can lead to suboptimal adaptation. Experiments on standard image benchmarks and a network intrusion detection task demonstrate that MaxDIRep achieves strong performance, works with pretrained models, and generalizes to non-image classification tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 47 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

26 of 47 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  4. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  5. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  6. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  7. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  8. 2017Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 682
  9. 2018On Lazy Training in Differentiable Programmingcited by 671
  10. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  11. 2016Domain Separation Networkscited by 317
  12. 2018Multi-Adversarial Domain Adaptationcited by 195
14 more
  1. 2017Generate To Adapt: Aligning Domains using Generative Adversarial Networkscited by 183
  2. 2019Domain Agnostic Learning with Disentangled Representationscited by 107
  3. 2022Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectivescited by 98
  4. 2017Unsupervised Domain Adaptation with Similarity Learningcited by 73
  5. 2020Adversarial-Learned Loss for Domain Adaptationcited by 65
  6. 2020Learning Disentangled Semantic Representation for Domain Adaptationcited by 54
  7. 2019Transfer Adaptation Learning: A Decade Surveycited by 52
  8. 2021Spectral Unsupervised Domain Adaptation for Visual Recognitioncited by 40
  9. 2021A Survey of Unsupervised Domain Adaptation for Visual Recognitioncited by 31
  10. 2020Variational Interaction Information Maximization for Cross-domain Disentanglementcited by 23
  11. 2021Variational Disentanglement for Domain Generalizationcited by 18
  12. 2021Information-theoretic regularization for Multi-source Domain Adaptationcited by 15
  13. 2019Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approachcited by 13
  14. 2018Improved Techniques for Adversarial Discriminative Domain Adaptationcited by 10

Built on by

Not cited yet.