mlsanityData to 13 Oct 2026

Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization

arXiv 2603.12369 ↗First posted 12 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Generalizing image classification across domains remains challenging in critical tasks such as fundus image-based diabetic retinopathy (DR) grading and resting-state fMRI seizure onset zone (SOZ) detection. When domains differ in unknown causal factors, achieving cross-domain generalization is difficult, and there is no established methodology to objectively assess such differences without direct metadata or protocol-level information from data collectors, which is typically inaccessible. We first introduce domain conformal bounds (DCB), a theoretical framework to evaluate whether domains diverge in unknown causal factors. Building on this, we propose GenEval, a multimodal Vision Language Models (VLM) approach that combines foundational models (e.g., MedGemma-4B) with human knowledge via Low-Rank Adaptation (LoRA) to bridge causal gaps and enhance single-source domain generalization (SDG). Across eight DR and two SOZ datasets, GenEval achieves superior SDG performance, with average accuracy of 69.2% (DR) and 81% (SOZ), outperforming the strongest baselines by 9.4% and 1.8%, respectively.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 42 of its 91 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

42 of 91 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,028
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,457
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,021
  6. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,324
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,296
  8. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 786
  9. 2019Conformal Prediction Under Covariate Shiftcited by 513
  10. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  11. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 389
  12. 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 302
30 more
  1. 2023A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAScited by 150
  2. 2023Deep Industrial Image Anomaly Detection: A Surveycited by 147
  3. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  4. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 140
  5. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
  6. 2022Understanding Gradient Descent on Edge of Stability in Deep Learningcited by 128
  7. 2020SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamicscited by 125
  8. 2024HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuningcited by 120
  9. 2021PySINDy: A comprehensive Python package for robust sparse system identificationcited by 88
  10. 2021Fair Conformal Predictors for Applications in Medical Imagingcited by 74
  11. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 63
  12. 2019A unified sparse optimization framework to learn parsimonious physics-informed models from datacited by 49
  13. 2020A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferabilitycited by 48
  14. 2023Domain Generalization for Medical Image Analysis: A Reviewcited by 48
  15. 2022Guided Depth Super-Resolution by Deep Anisotropic Diffusioncited by 46
  16. 2022OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regressioncited by 44
  17. 2021KL Guided Domain Adaptationcited by 35
  18. 2022Self-Distilled Vision Transformer for Domain Generalizationcited by 35
  19. 2019Learn to Segment Retinal Lesions and Beyondcited by 33
  20. 2015On learning optimized reaction diffusion processes for effective image restorationcited by 25
  21. 2023Towards Generalizable Diabetic Retinopathy Grading in Unseen Domainscited by 21
  22. 2024Anatomical Structure-Guided Medical Vision-Language Pre-trainingcited by 18
  23. 2024CLIP-DR: Textual Knowledge-Guided Diabetic Retinopathy Grading with Ranking-aware Promptingcited by 18
  24. 2024Generalizing to Unseen Domains in Diabetic Retinopathy with Disentangled Representationscited by 11
  25. 2024DUDF: Differentiable Unsigned Distance Fields with Hyperbolic Scalingcited by 9
  26. 2023Generalizing to Unseen Domains in Diabetic Retinopathy Classificationcited by 7
  27. 2025Pitfalls of Conformal Predictions for Medical Image Classificationcited by 6
  28. 2025Generating customized prompts for Zero-Shot Rare Event Medical Image Classification using LLMcited by 2
  29. 2025Why Domain Generalization Fail? A View of Necessity and Sufficiencycited by 2
  30. 2024Physics Meets Pixels: PDE Models in Image Processingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Single Domain Generalization in Diabetic Retinopathy: A Neuro-Symbolic Learning Approach
  2. 2024PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization
  3. 2025Latent Domain Prompt Learning for Vision-Language Models
  4. 2025Beyond CLIP: Knowledge-Enhanced Multimodal Transformers for Cross-Modal Alignment in Diabetic Retinopathy Diagnosis
  5. 2026FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs
  6. 2024DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization
  7. 2025MedXAI: A Retrieval-Augmented and Self-Verifying Framework for Knowledge-Guided Medical Image Analysis
  8. 2024Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research