mlsanityData to 14 Oct 2026

QwenVLConnector: A Fast, Unified Medical VLM Chatbot for Fine-Grained Clinical Perception and Text Generation

arXiv 2609.23139 ↗First posted 19 Sep 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Most medical vision-language models (VLMs) excel at open-ended report generation and VQA but provide limited support for structured, fine-grained clinical perception within a unified interface. We present QwenVLConnector, a Qwen2.5-VL-based medical chatbot that unifies classification, multi-label classification, textualized detection, counting, regression, and free-form report generation under a single next-token objective. Our key component is a lightweight dense multi-layer Connector that aggregates low- and high-level visual features, aligns them through the pretrained vision Merger, and fuses them with the final visual representation without increasing sequence length. This design enriches visual tokens with complementary spatial and semantic cues while preserving efficiency. On FLARE-2D, QwenVLConnector improves detection F1 from 0.55 to 0.85, raises single-label classification from 0.37 to 0.51, and boosts report-generation GREEN by up to 18.3 points over the Qwen2.5-VL baseline. We further explore multimodal in-context learning for report generation, showing additional improvements without updating model parameters. Overall, QwenVLConnector offers a unified and efficient framework for combining structured medical perception with open-ended clinical text generation. Our code can be found at https://github.com/plnguyen2908/QwenConnector.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 32 references that are in the map. First posted on arXiv 19 Sep 2026.

Builds on

25 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis25 Mar
  2. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  3. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 22

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2015VQA: Visual Question Answeringcited by 2,506
  5. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  7. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  8. 2018Object Detection with Deep Learning: A Reviewcited by 528
  9. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  10. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 334
  11. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 214
  12. 2024Many-Shot In-Context Learningcited by 206
10 more
  1. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
  2. 2020Metrics for Multi-Class Classification: an Overviewcited by 143
  3. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  4. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  5. 2024GREEN: Generative Radiology Report Evaluation and Error Notationcited by 128
  6. 2021Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platformcited by 66
  7. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
  8. 2025See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Modelscited by 20
  9. 2025Revisiting Active Speaker Detection: An In-the-Wild Benchmark for Generalization and Robustnesscited by 6
  10. 2026DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documentscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering
  2. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
  4. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding
  5. 2024Advancing High Resolution Vision-Language Models in Biomedicine
  6. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
  7. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
  8. 2025Fleming-VL: Towards Universal Medical Visual Reasoning with Multimodal LLMs