mlsanityData to 14 Oct 2026

Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift

arXiv 2607.28696 ↗First posted 30 Jul 2026cs.LG cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical vision-language models (VLMs) can retain high observed marginal coverage after clinical shift while substantially under-covering an individual disease class. The affected class varies with acquisition protocol and backbone geometry, so source prevalence does not reliably reveal the failure. Existing localized and tail-aware conformal methods respectively adapt to test neighborhoods and source-frequency tails, leaving held-out class-wise coverage failure unmodeled. We introduce Class-Tail Adaptive Localized Conformal Deferral (CALCoDe), a post-hoc reliability layer for frozen medical VLMs. Cross-fitted validation predictions identify classes at risk of undercoverage, and a disjoint calibration split estimates their class-conditional tail thresholds. CALCoDe combines each protected threshold with a localized conformal threshold using a one-sided maximum. The resulting set contains every label admitted by the localized rule, with additional protection confined to validation-identified classes. An independently calibrated support audit defers cases with insufficient inlier support. Under exchangeability among accepted examples within each protected class, CALCoDe provides finite-sample coverage at the prespecified guard level and contains the corresponding localized conformal sets; coverage on shifted external cohorts is evaluated empirically. Among standard conformal baselines and recent VLM-specific conformal methods evaluated across two dermatology shifts (HAM10000 to ISIC 2019 and HAM10000 to PAD-UFES-20) and four frozen VLM backbones (BiomedCLIP, OpenAI CLIP ViT-B/32, PubMedCLIP ViT-B/32, and MedSigLIP-448), CALCoDe is the only approach whose observed marginal and worst-class accepted coverage both reach 0.95 in all eight settings. On HAM10000 to ISIC 2019, its average worst-class accepted coverage is 0.970, compared with 0.926 for sTACP and 0.864 for LCP-VLM.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 28 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

21 of 28 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025vMFCoOp: Towards Equilibrium on a Unified Hyperspherical Manifold for Prompting Biomedical VLMs12 Nov
  2. 2025Calibration-Aware Prompt Learning for Medical Vision-Language Models18 Sep
  3. 2024BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models21 Nov
  4. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine6 Aug
  5. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models10 Jun

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  4. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  5. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  6. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  7. 2019Conformal Prediction Under Covariate Shiftcited by 519
  8. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
  9. 2021Localized Conformal Prediction: A Generalized Inference Framework for Conformal Predictioncited by 114
  10. 2023Conformal prediction with local weights: randomization enables local guaranteescited by 48
  11. 2019Prediction and outlier detection in classification problemscited by 44
  12. 2025Conformal Prediction for Zero-Shot Modelscited by 15
4 more
  1. 2025OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detectioncited by 14
  2. 2025Conformal Prediction Meets Long-tail Classificationcited by 5
  3. 2026Empirical Bayes Conformal Prediction for Vision and Language Modelscited by 1
  4. 2026Localized Conformal Prediction for Image Classification with Vision-Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Full Conformal Adaptation of Medical Vision-Language Models
  2. 2026Learning to Adapt and Calibrate: Score Distribution Alignment for Few-Shot Uncertainty Prediction in Medical VLMs
  3. 2026LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs
  4. 2025Trustworthy Few-Shot Transfer of Medical VLMs through Split Conformal Prediction
  5. 2026TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
  6. 2026A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks
  7. 2026Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?
  8. 2025Decoupling Clinical and Class-Agnostic Features for Reliable Few-Shot Adaptation under Shift