# PromptGate Client Adaptive Vision Language Gating for Open Set Federated Active Learning

arXiv: https://arxiv.org/abs/2603.07163 · HTML: https://mlsanity.com/paper/2603.07163/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 7 Mar 2026.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset](https://mlsanity.com/paper/2104.09957/) (2021)
- [Disparities in Dermatology AI: Assessments Using Diverse Clinical Images](https://mlsanity.com/paper/2111.08006/) (2021)
- [FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings](https://mlsanity.com/paper/2210.04620/) (2022)
- [Active Learning for Open-set Annotation](https://mlsanity.com/paper/2201.06758/) (2022)
- [Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts](https://mlsanity.com/paper/2312.02567/) (2023)
- [Robust image representations with counterfactual contrastive learning](https://mlsanity.com/paper/2409.10365/) (2024)
- [Joint Out-of-Distribution Filtering and Data Discovery Active Learning](https://mlsanity.com/paper/2503.02491/) (2025)
- [OpenAL: An Efficient Deep Active Learning Framework for Open-Set Pathology Image Classification](https://mlsanity.com/paper/2307.05254/) (2023)
- [Radio-opaque artefacts in digital mammography: automatic detection and analysis of downstream effects](https://mlsanity.com/paper/2410.03809/) (2024)
- [Bias and Generalizability of Foundation Models across Datasets in Breast Mammography](https://mlsanity.com/paper/2505.10579/) (2025)
- [OpenPath: Open-Set Active Learning for Pathology Image Classification via Pre-trained Vision-Language Models](https://mlsanity.com/paper/2506.15318/) (2025)

## Abstract (arXiv)

Deploying medical AI across resource-constrained institutions demands data-efficient learning pipelines that respect patient privacy. Federated Learning (FL) enables collaborative medical AI without centralising data, yet real-world clinical pools are inherently open-set, containing out-of-distribution (OOD) noise such as imaging artifacts and wrong modalities. Standard Active Learning (AL) query strategies mistake this noise for informative samples, wasting scarce annotation budgets. We propose PromptGate, a dynamic VLM-gated framework for Open-Set Federated AL that purifies unlabeled pools before querying. PromptGate introduces a federated Class-Specific Context Optimization: lightweight, learnable prompt vectors that adapt a frozen BiomedCLIP backbone to local clinical domains and aggregate globally via FedAvg -- without sharing patient data. As new annotations arrive, prompts progressively sharpen the ID/OOD boundary, turning the VLM into a dynamic gatekeeper that is strategy-agnostic: a plug-and-play pre-selection module enhancing any downstream AL strategy. Experiments on distributed dermatology and breast imaging benchmarks show that while static VLM prompting degrades to 50% ID purity, PromptGate maintains >95% purity with 98% OOD recall.