mlsanityData to 14 Oct 2026

Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters

arXiv 2610.19901 ↗First posted 14 Oct 2026cs.CV cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Adapting a single vision-language model to medical image parsing requires training choices that reflect the demands of each task. We study task-specific adaptation with Qwen3.5-9B for FLARE 2026 Task 3, covering six task families and seven imaging modalities. Our system couples a shared, frozen 4-bit backbone with LoRA adapters selected from the input question. We customize adapter ranks and prompts, rebalance rare classification targets, and supplement regression training with dermatology classification examples. On the fixed internal holdout, the task-specific adapters achieve a dataset-macro balanced accuracy of 0.7791 for classification, a dataset-macro micro-F1 of 0.6286 for multi-label classification, a counting mean absolute error (MAE) of 11.73, a dataset-macro detection F1 of 0.8914, a dataset-macro regression MAE of 9.78, and a report-generation CRIMSON score of 0.8204. On the organizer's 1,703-example hidden-validation set, the submitted system achieves 0.8496 classification balanced accuracy, 0.5338 sample-averaged multi-label F1, and 14.5950 regression MAE. The final 30.1 GB container averages 3.1 seconds per case, including model loading, and produces identical outputs across repeated verification runs. The benefits vary across tasks, supporting the use of task-specific adapter configurations and image budgets within a shared medical vision-language model. Code is available at https://github.com/TahaSabir0/FLARE26-Hexapus.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 23 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

15 of 23 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026QwenVLConnector: A Fast, Unified Medical VLM Chatbot for Fine-Grained Clinical Perception and Text Generation19 Sep
  2. 2025Can Generalist Vision Language Models (VLMs) Rival Specialist Medical VLMs? Benchmarking and Strategic Insights19 Jun
  3. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models16 Apr
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 11

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  7. 2025MedGemma Technical Reportcited by 436
  8. 2021Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platformcited by 66
  9. 2025GLiClass: Generalist Lightweight Model for Sequence Classification Taskscited by 15
  10. 2026CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluationcited by 9
  11. 2025Understanding Task Transfer in Vision-Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Two-Stage Mixture-of-LoRA for Multi-Task Medical Vision-Language Learning
  2. 2024Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training
  3. 2025Point, Detect, Count: Multi-Task Medical Image Understanding with Instruction-Tuned Vision-Language Models
  4. 2024Few-shot Adaptation of Medical Vision-Language Models
  5. 2026LLM is Not All You Need: A Systematic Evaluation of ML vs. Foundation Models for text and image based Medical Classification
  6. 2025Medical Knowledge Intervention Prompt Tuning for Medical Image Classification
  7. 2026How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective
  8. 2025QwenCLIP: Boosting Medical Vision-Language Pretraining via LLM Embeddings and Prompt tuning