mlsanityData to 7 Oct 2026

Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding

arXiv 2510.08668 ↗First posted 9 Oct 2025cs.CVcited by 96

Not on the map. Why?

Cited by 46 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Real-world clinical decision-making requires integrating heterogeneous data, including medical text, 2D images, 3D volumes, and videos, while existing AI systems fail to unify all these signals, limiting their utility. In this paper, we introduce Hulu-Med, a transparent, generalist medical Vision-Language Model (VLM) designed to unify language-only, 2D/3D vision-language, and video understanding within a single architecture. Hulu-Med is trained on a curated corpus of 16.7 million samples, comprising exclusively public or synthetic data, spanning 12 major anatomical systems and 14 medical imaging modalities. Hulu-Med employs a medical-aware token-reduction strategy that prunes redundant visual tokens, achieving up to a 55% reduction for 3D and video inputs, improving cross-modal efficiency, and enabling training at 7B-32B parameter scales in approximately 4,000-40,000 GPU hours. Across 30 public in-domain and out-of-domain medical benchmarks-covering text reasoning, visual question answering, report generation, multilingual dialogue, video understanding, and rare disease diagnosis-Hulu-Med surpasses existing open-source models on 27 of 30 benchmarks and outperforms proprietary systems such as GPT-4o on 16 benchmarks. Despite being a VLM, Hulu-Med outperforms GPT-4o and matches GPT-o1 on the text-only HealthBench. For the first time in the community, we provide a fully transparent, reproducible and cost-effective pipeline for holistic medical vision-language understanding by releasing our end-to-end data curation, training procedures, and model parameters. Code and models are available at https://github.com/ZJUI-AI4H/Hulu-Med.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Oct 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 91.

96 papers.

Sort:

By area:

  1. 2025CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimizationcited by 24
  2. 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
  3. 2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 10
  4. 2026Real Money, Fake Models: Deceptive Model Claims in Shadow APIscited by 8
  5. 2026MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learningcited by 6
  6. 2026Brain-WM: Brain Glioblastoma World Modelcited by 5
  7. 2026When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Modelscited by 5
  8. 2026TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agentscited by 4
  9. 2026Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicinecited by 4
  10. 2025Trustworthy and Fair SkinGPT-R1 for Democratizing Dermatological Reasoning across Diverse Ethnicitiescited by 3
  11. 2026Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Benchcited by 3
  12. 2026CodePercept: Code-Grounded Visual STEM Perception for MLLMscited by 3
38 more
  1. 2026Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMscited by 3
  2. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 3
  3. 2026Medical thinking with multiple imagescited by 3
  4. 2026MedHorizon: Towards Long-context Medical Video Understanding in the Wildcited by 3
  5. 2026MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoningcited by 3
  6. 2026OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysiscited by 3
  7. 2026Generating Reports or Repeating Templates? Measuring and Mitigating Template Collapse in 3D CT Report Generationcited by 3
  8. 2026Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policiescited by 3
  9. 2026Can Multimodal Large Language Models Understand OCT?cited by 3
  10. 2025Radiologist Copilot: An Agentic Framework Orchestrating Specialized Tools for Reliable Radiology Reportingcited by 2
  11. 2026MM-NeuroOnco: A Multimodal Benchmark and Instruction Dataset for MRI-Based Brain Tumor Diagnosiscited by 2
  12. 2026ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretationcited by 2
  13. 2026Learning Transferable Temporal Primitives for Video Reasoning via Synthetic Videoscited by 2
  14. 2026MedRCube: A Multidimensional Framework for Fine-Grained and In-Depth Evaluation of MLLMs in Medical Imagingcited by 2
  15. 2026How Far Are Video Models from True Multimodal Reasoning?cited by 2
  16. 2026Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysiscited by 2
  17. 2026ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understandingcited by 2
  18. 2025CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Trainingcited by 1
  19. 2026KorMedMCQA-V: A Multimodal Benchmark for Evaluating Vision-Language Models on the Korean Medical Licensing Examinationcited by 1
  20. 2026SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Videocited by 1
  21. 2026CT-Flow: Orchestrating CT Interpretation Workflow with Model Context Protocol Serverscited by 1
  22. 2026CORE-Seg: Reasoning-Driven Segmentation for Complex Lesions via Reinforcement Learningcited by 1
  23. 2026RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Reviewcited by 1
  24. 2026MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Modelscited by 1
  25. 2026IOSVLM: A 3D Vision-Language Model for Unified Dental Diagnosis from Intraoral Scanscited by 1
  26. 2026Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classificationcited by 1
  27. 2026MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmarkcited by 1
  28. 2026CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generationcited by 1
  29. 2026SurgOnAir: Hierarchy-Aware Real-Time Surgical Video Commentarycited by 1
  30. 2026Mind the Tool Failures: Achieving Synergistic Tool Gains for Medical Agentscited by 1
  31. 2026A Vision-language Framework for Comparative Reasoning in Radiologycited by 1
  32. 2026Baichuan-M4: A Clinical-Grade Medical Agent System for Continuous Carecited by 1
  33. 2026ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoningcited by 1
  34. 2026ReportQA: QA-Based Radiology Report Evaluationcited by 1
  35. 2026CheXpercept: A Benchmark for Evaluating Expert-Level Lesion Perception in Chest X-rayscited by 1
  36. 2026Aloe-Vision: Robust Vision-Language Models for Healthcarecited by 1
  37. 2026RadSight: Towards Perceptually Reliable Multimodal Radiology Image Understandingcited by 1
  38. 2026Sparse Multi-Stage Expert-Agent Routing for Complex Clinical Reasoningcited by 1

50 most cited of 96 All 96 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025How Far Have Medical Vision-Language Models Come? A Comprehensive Benchmarking Study
  2. 2025OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding
  3. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
  4. 2024Beyond the Hype: A dispassionate look at vision-language models in medical scenario
  5. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
  6. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning
  7. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
  8. 2025Fleming-VL: Towards Universal Medical Visual Reasoning with Multimodal LLMs