mlsanityData to 14 Oct 2026

Addressing Bias in VLMs for Glaucoma Detection Without Protected Attribute Supervision

arXiv 2508.09087 ↗First posted 12 Aug 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs) have achieved remarkable success on multimodal tasks such as image-text retrieval and zero-shot classification, yet they can exhibit demographic biases even when explicit protected attributes are absent during training. In this work, we focus on automated glaucoma screening from retinal fundus images, a critical application given that glaucoma is a leading cause of irreversible blindness and disproportionately affects underserved populations. Building on a reweighting-based contrastive learning framework, we introduce an attribute-agnostic debiasing method that (i) infers proxy subgroups via unsupervised clustering of image-image embeddings, (ii) computes gradient-similarity weights between the CLIP-style multimodal loss and a SimCLR-style image-pair contrastive loss, and (iii) applies these weights in a joint, top-k weighted objective to upweight underperforming clusters. This label-free approach adaptively targets the hardest examples, thereby reducing subgroup disparities. We evaluate our method on the Harvard FairVLMed glaucoma subset, reporting Equalized Odds Distance (EOD), Equalized Subgroup AUC (ES AUC), and Groupwise AUC to demonstrate equitable performance across inferred demographic subgroups.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 20 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

10 of 20 references.

References · 10

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  4. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 672
  5. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 364
  6. 2024FairCLIP: Harnessing Fairness in Vision-Language Learningcited by 69
  7. 2024A Unified Debiasing Approach for Vision-Language Models across Modalities and Taskscited by 18
  8. 2023An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Modelscited by 16
  9. 2023Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Modelscited by 12
  10. 2024BMFT: Achieving Fairness via Bias-based Weight Masking Fine-tuningcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FairEnc: A Fair Vision-Language Model with Fair Vision and Text Encoders for Glaucoma Detection
  2. 2025Fairness-Aware Fine-Tuning of Vision-Language Models for Medical Glaucoma Diagnosis
  3. 2026FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs
  4. 2026Balanced Soft mixture-of-expert model for Glaucoma Detection
  5. 2026An Agentic AI Framework Overcomes Fundamental Limitations of Large Language Models for Glaucoma Detection from Fundus Photography
  6. 2025Interpretable Few-Shot Retinal Disease Diagnosis with Concept-Guided Prompting of Vision-Language Models
  7. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision
  8. 2025RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Models