mlsanityData to 14 Oct 2026

Two-Stage Mixture-of-LoRA for Multi-Task Medical Vision-Language Learning

arXiv 2609.14350 ↗First posted 13 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical vision-language models (VLMs) allow a single model to perform clinical image analysis tasks ranging from diagnosis classification to report generation. However, joint adaptation is challenged by heterogeneous output formats, conflicting task gradients, and imbalanced training data. Hence, we present Two-Stage Mixture-of-LoRA, a framework built on MedGemma-1.5-4B. The framework uses a shared-specific Mixture-of-LoRA architecture comprising one shared LoRA and six task-specific expert LoRAs, together with a two-stage training procedure. In Stage 1, we jointly train the shared LoRA and all task-specific expert LoRAs on all tasks. In Stage 2, we first freeze the backbone, the shared LoRA, and all non-target experts, and refine one task expert at a time. Classification and regression then receive an additional modality-balanced continuation, in which smaller modality groups are repeated to match the largest group. In the FLARE 2026 Task 3 test sets, the proposed method achieves 0.85 balanced accuracy for classification, 0.48 micro-F1 for multi-label classification, 0.79 detection F1, and 17.39 regression MAE. Code is available at https://github.com/YuanYL03/MICCAI-FLARE-2026-Challenge-Task3-2D.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 19 references that are in the map. First posted on arXiv 13 Sep 2026.

Builds on

14 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 12

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2025Qwen3-VL Technical Reportcited by 2,793
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  7. 2024Mixture of LoRA Expertscited by 181
  8. 2026Qwen3.5-Omni Technical Reportcited by 174
  9. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 102
  10. 2021Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platformcited by 66
  11. 2026MedGemma 1.5 Technical Reportcited by 45
  12. 2026CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluationcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Model Merging for Medical LVLMs: A Benchmark and a Winner-Take-All Approach
  2. 2026Sparse Spectral LoRA: Routed Experts for Medical VLMs
  3. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
  4. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review
  5. 2026Mechanistically Guided LoRA Improves Paraphrase Consistency in Medical Vision-Language Models
  6. 2026CADRE: Stable, Parameter Efficient Adaptation of Medical Vision Language Models with Bounded Forgetting and Prior Drift
  7. 2023UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts
  8. 2025Scaling Down to Scale Up: Towards Operationally-Efficient and Deployable Clinical Models via Cross-Modal Low-Rank Adaptation for Medical Vision-Language Models