HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
also in 1 more
Abstract (arXiv)
Medical Vision-Language Models (Med-VLMs) have achieved success across various tasks, yet most existing methods overlook the modality misalignment issue that can lead to untrustworthy responses in clinical settings. In this paper, we propose Hierarchical Self-Contrastive Rewarding (HSCR), a novel approach that addresses two critical challenges in Med-VLM alignment: 1) Cost-effective generation of high-quality preference data; 2) Capturing nuanced and context-aware preferences for improved alignment. HSCR first leverages the inherent capability of Med-VLMs to generate dispreferred responses with higher sampling probability. By analyzing output logit shifts after visual token dropout, we identify modality-coupled tokens that induce misalignment and derive an implicit alignment reward function. This function guides token replacement with hallucinated ones during decoding, producing high-quality dispreferred data. Furthermore, HSCR introduces a multi-level preference optimization strategy, which extends beyond traditional adjacent-level optimization by incorporating nuanced implicit preferences, leveraging relative quality in dispreferred data to capture subtle alignment cues for more precise and context-aware optimization. Extensive experiments across multiple medical tasks, including Med-VQA, medical image captioning and instruction following, demonstrate that HSCR not only enhances zero-shot performance but also significantly improves modality alignment and trustworthiness with just 2,000 training entries.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 61 references that are in the map. First posted on arXiv 1 Jun 2025.
Builds on
35 of 61 references.
Closest ancestors · 3
Same direction, newest first
- 2025OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding20 Apr
- 2024STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering28 Jun
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun
Other references · 32
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
20 more
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
- 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
- 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
- 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 300
- 2023Preference Ranking Optimization for Human Alignmentcited by 285
- 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 241
- 2024Capabilities of Gemini Models in Medicinecited by 237
- 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
- 2023Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimizationcited by 124
- 2024LiPO: Listwise Preference Optimization through Learning-to-Rankcited by 96
- 2024Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimizationcited by 75
- 2023Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Modelscited by 58
- 2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
- 2024MedCoT: Medical Chain of Thought via Hierarchical Expertcited by 46
- 2023Q2ATransformer: Improving Medical VQA via an Answer Querying Decodercited by 31
- 2024Automated Multi-level Preference for MLLMscited by 29
- 2024Modality-Fair Preference Optimization for Trustworthy MLLM Alignmentcited by 26
- 2024SQ-LLaVA: Self-Questioning for Large Vision-Language Assistantcited by 18
- 2023Stabilizing RLHF through Advantage Model and Selective Rehearsalcited by 14
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs
- 2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
- 2024Calibrated Self-Rewarding Vision Language Models
- 2024Synth-Align: Improving Trustworthiness in Vision-Language Model with Synthetic Preference Data Alignment
- 2024Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment
- 2025Token-Level Inference-Time Alignment for Vision-Language Models
- 2025Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization
- 2025Enhancing Medical Large Vision-Language Models via Alignment Distillation