# CoFiDA-M: Concept-Aware Feature Modulation for Cross-Domain Adaptation with Image-Only Inference

arXiv: https://arxiv.org/abs/2605.31591 · HTML: https://mlsanity.com/paper/2605.31591/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 29 May 2026.

## Builds on (most cited first, 26 of 26)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer](https://mlsanity.com/paper/2012.07297/) (2020)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [PODA: Prompt-driven Zero-shot Domain Adaptation](https://mlsanity.com/paper/2212.03241/) (2022)
- [DINE: Domain Adaptation from Single and Multiple Black-box Predictors](https://mlsanity.com/paper/2104.01539/) (2021)
- [A Multimodal Vision Foundation Model for Clinical Dermatology](https://mlsanity.com/paper/2410.15038/) (2024)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.02899/) (2024)
- [TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis](https://mlsanity.com/paper/2211.11393/) (2022)
- [MIMO: A medical vision language model with visual referring multimodal input and pixel grounding multimodal output](https://mlsanity.com/paper/2510.10011/) (2025)
- [MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbooks](https://mlsanity.com/paper/2505.06152/) (2025)
- [MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment](https://mlsanity.com/paper/2505.09372/) (2025)
- [Achieving Reliable and Fair Skin Lesion Diagnosis via Unsupervised Domain Adaptation](https://mlsanity.com/paper/2307.03157/) (2023)
- [Unsupervised domain adaptation by learning using privileged information](https://mlsanity.com/paper/2303.09350/) (2023)
- [IT-RUDA: Information Theory Assisted Robust Unsupervised Domain Adaptation](https://mlsanity.com/paper/2210.12947/) (2022)
- [Skin Lesion Phenotyping via Nested Multi-modal Contrastive Learning](https://mlsanity.com/paper/2505.23709/) (2025)

## Abstract (arXiv)

Models for AI-based skin cancer screening suffer a severe performance drop when shifting from expert dermoscopic (source) images to consumer-grade clinical (target) images, hindering real-world deployment. Existing domain adaptation methods often ignore crucial semantic invariants, such as clinical concepts. While new foundation models like MONET can provide this semantic information as dense, probabilistic scores, this metadata is unavailable at test time, creating a deployment paradox for practical image-only screening tools. We address this gap by proposing CoFiDA-M, a privileged information framework that learns from concepts at training time but deploys as an image-only model. Our method trains a teacher network that uses MONET concept probabilities to guide a FiLM modulator, transforming visual features into a semantically ``edited" feature space. A lightweight, image-only student is then trained to reproduce this edited representation, not just the teacher's final predictions. This distillation ``bakes" the clinical reasoning into the student's weights. On a challenging multi-dataset benchmark, our image-only student significantly outperforms state-of-the-art approaches, especially in melanoma recall. Our work provides a practical and generalizable framework for leveraging noisy, probabilistic metadata as privileged information, demonstrating strong cross-dataset robustness and potential for real-world deployment beyond dermatology. Implementation code is available at: https://github.com/mmu-dermatology-research/CoFiDA.git