mlsanityData to 14 Oct 2026

PatchGen: Learning Soft Intra-Image Predictive Subsets for Visual Generalization

arXiv 2608.12766 ↗First posted 13 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual classifiers are expected to generalize under data shifts, target shifts, and their combinations, yet most existing methods focus on domain invariance while failing to address intra-image predictive sufficiency. We investigate the structural hypothesis that each image contains a sample-adaptive oracle intra-image predictive subset sufficient for label prediction, while the remaining patches form non-essential complementary context that may correlate with the label. The theoretical analysis shows that restricting prediction to this oracle subset preserves the Bayes risk achievable by the full-patch representation while admitting a complexity bound that tightens with the oracle-subset size. Based on this view, we propose PatchGen, a text-free module that learns a sample-dependent soft predictive-subset mask as a task-driven proxy for the unobserved oracle subset mask. Specifically, histopathology visualizations suggest that PatchGen assigns higher scores to tumor-consistent regions than to some frequently co-occurring inflammatory context. Extensive experiments on natural and histopathological image benchmarks spanning all three shift settings show that PatchGen improves average performance over matched-backbone baselines in most evaluated configurations, enhances generalization to unknown classes, and remains competitive with vision-language methods without text supervision.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 58 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

37 of 58 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2019Invariant Risk Minimizationcited by 1,854
  6. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  7. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  8. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  9. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  10. 2020In Search of Lost Domain Generalizationcited by 1,054
  11. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  12. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
25 more
  1. 2020Quantifying Attention Flow in Transformerscited by 775
  2. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  3. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  4. 2018Recognition in Terra Incognitacited by 658
  5. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
  6. 2021Gradient Matching for Domain Generalizationcited by 258
  7. 2021SelfReg: Self-supervised Contrastive Regularization for Domain Generalizationcited by 189
  8. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  9. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  10. 2018Domain Generalization via Conditional Invariant Representationcited by 146
  11. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  12. 2023CLIPood: Generalizing CLIP to Out-of-Distributionscited by 96
  13. 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
  14. 2019Domain Generalization via Multidomain Discriminant Analysiscited by 69
  15. 2023PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalizationcited by 59
  16. 2021Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domainscited by 39
  17. 2022Grow and Merge: A Unified Framework for Continuous Categories Discoverycited by 33
  18. 2024Rethinking Multi-domain Generalization with A General Learning Objectivecited by 23
  19. 2022Domain-Unified Prompt Representations for Source-Free Domain Generalizationcited by 21
  20. 2021Isometric Propagation Network for Generalized Zero-shot Learningcited by 18
  21. 2021Domain-Specific Bias Filtering for Single Labeled Domain Generalizationcited by 18
  22. 2023On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformerscited by 14
  23. 2023Domain Generalization Using Large Pretrained Models with Mixture-of-Adapterscited by 11
  24. 2024Interpret Your Decision: Logical Reasoning Regularization for Generalization in Visual Classificationcited by 5
  25. 2025Exploiting Layer Normalization Fine-tuning in Visual Transformer Foundation Models for Classificationcited by 3

Built on by

Not cited yet.