mlsanityData to 7 Oct 2026

Domain Generalization by Mutual-Information Regularization with Pre-trained Models

arXiv 2203.10789 ↗First posted 21 Mar 2022cs.LG cs.CVcited by 133

Not on the map. Why?

Abstract (arXiv)

Domain generalization (DG) aims to learn a generalized model to an unseen target domain using only limited source domains. Previous attempts to DG fail to learn domain-invariant representations only from the source domains due to the significant domain shifts between training and test domains. Instead, we re-formulate the DG objective using mutual information with the oracle model, a model generalized to any possible domain. We derive a tractable variational lower bound via approximating the oracle model by a pre-trained model, called Mutual Information Regularization with Oracle (MIRO). Our extensive experiments show that MIRO significantly improves the out-of-distribution performance. Furthermore, our scaling experiments show that the larger the scale of the pre-trained model, the greater the performance improvement of MIRO. Source code is available at https://github.com/kakaobrain/miro.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Builds on 63 of its 76 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

63 of 76 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  7. 2016Learning without Forgettingcited by 2,465
  8. 2019Invariant Risk Minimizationcited by 1,832
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  12. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
38 more
  1. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  2. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  3. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  4. 2020In Search of Lost Domain Generalizationcited by 1,042
  5. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  6. 2020Designing Network Design Spacescited by 954
  7. 2021Robust fine-tuning of zero-shot modelscited by 785
  8. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  9. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  10. 2019Contrastive Representation Distillationcited by 734
  11. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  12. 2018MINE: Mutual Information Neural Estimationcited by 680
  13. 2018Recognition in Terra Incognitacited by 655
  14. 2013Domain Generalization via Invariant Feature Representationcited by 622
  15. 2021Domain Generalization with MixStylecited by 563
  16. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  17. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  18. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  19. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  20. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  21. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  22. 2019Variational Information Distillation for Knowledge Transfercited by 321
  23. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  24. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  25. 2021Gradient Matching for Domain Generalizationcited by 258
  26. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  27. 2019Episodic Training for Domain Generalizationcited by 245
  28. 2018Explicit Inductive Bias for Transfer Learning with Convolutional Networkscited by 226
  29. 2019Learning De-biased Representations with Biased Representationscited by 199
  30. 2021SelfReg: Self-supervised Contrastive Regularization for Domain Generalizationcited by 189
  31. 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
  32. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  33. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  34. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  35. 2019Domain Generalization Using a Mixture of Multiple Latent Domainscited by 170
  36. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  37. 2019Does Object Recognition Work for Everyone?cited by 149
  38. 2018Domain Generalization via Conditional Invariant Representationcited by 146

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 47; 2024: 36; 2025: 24; 2026: 16.

133 papers.

Sort:

By area:

  1. 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
  2. 2023Sharpness-Aware Gradient Matching for Domain Generalizationcited by 102
  3. 2023CLIPood: Generalizing CLIP to Out-of-Distributionscited by 96
  4. 2022Sparse Mixture-of-Experts are Domain Generalizable Learnerscited by 85
  5. 2023Nearest Neighbor Guidance for Out-of-Distribution Detectioncited by 64
  6. 2023PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalizationcited by 59
  7. 2023Read-only Prompt Optimization for Vision-Language Few-shot Learningcited by 58
  8. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  9. 2023A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidancecited by 40
  10. 2023StyLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based Domain Generalizationcited by 36
  11. 2024DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learningcited by 32
  12. 2022Exploring Visual Interpretability for Contrastive Language-Image Pre-trainingcited by 29
38 more
  1. 2022Context-Aware Robust Fine-Tuningcited by 27
  2. 2023Flatness-Aware Minimization for Domain Generalizationcited by 27
  3. 2023Generalizable Decision Boundaries: Dualistic Meta-Learning for Open Set Domain Generalizationcited by 24
  4. 2024Soft Prompt Generation for Domain Generalizationcited by 24
  5. 2024Rethinking Multi-domain Generalization with A General Learning Objectivecited by 23
  6. 2024Adapting to Distribution Shift by Visual Domain Prompt Generationcited by 23
  7. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  8. 2023MADG: Margin-based Adversarial Learning for Domain Generalizationcited by 22
  9. 2022Domain-Unified Prompt Representations for Source-Free Domain Generalizationcited by 21
  10. 2023DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalizationcited by 21
  11. 2023DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networkscited by 20
  12. 2023Rethinking the Evaluation Protocol of Domain Generalizationcited by 20
  13. 2024Proxy Denoising for Source-Free Domain Adaptationcited by 20
  14. 2023MI-SegNet: Mutual Information-Based US Segmentation for Unseen Domain Generalizationcited by 18
  15. 2023ERM++: An Improved Baseline for Domain Generalizationcited by 18
  16. 2023CHAMMI: A benchmark for channel-adaptive models in microscopy imagingcited by 18
  17. 2023Domain Generalization via Rationale Invariancecited by 16
  18. 2024PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalizationcited by 15
  19. 2024Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalizationcited by 14
  20. 2024HYPO: Hyperspherical Out-of-Distribution Generalizationcited by 13
  21. 2023Domain Generalization via Nuclear Norm Regularizationcited by 12
  22. 2023Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalizationcited by 12
  23. 2024Advancing Open-Set Domain Generalization Using Evidential Bi-Level Hardest Domain Schedulercited by 12
  24. 2023Domain Generalization Using Large Pretrained Models with Mixture-of-Adapterscited by 11
  25. 2022Domain Generalization with Correlated Style Uncertaintycited by 10
  26. 2023Diverse Target and Contribution Scheduling for Domain Generalizationcited by 10
  27. 2024Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapescited by 10
  28. 2023Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Modelscited by 9
  29. 2023Distilling Out-of-Distribution Robustness from Vision-Language Foundation Modelscited by 9
  30. 2024Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Modelscited by 9
  31. 2024Is Large-Scale Pretraining the Secret to Good Domain Generalization?cited by 9
  32. 2023POEM: Polarization of Embeddings for Domain-Invariant Representationscited by 8
  33. 2024HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalizationcited by 8
  34. 2024LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalizationcited by 8
  35. 2025Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalizationcited by 8
  36. 2025Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIPcited by 7
  37. 2025PEER pressure: Model-to-Model Regularization for Single Source Domain Generalizationcited by 7
  38. 2023Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimizationcited by 6

50 most cited of 133 All 133 (JSON)