mlsanityData to 7 Oct 2026

UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis

arXiv 2510.15710 ↗First posted 17 Oct 2025cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central to medical AI. Most existing systems, however, address these abilities in isolated models, losing the shared knowledge that a unified architecture could exploit. To bridge this gap, we present UniMedVL, the first unified medical model that seamlessly integrates multimodal understanding and generation capabilities within a single model without switching weights. We achieve this via a tailored progressive training pipeline where understanding and generation mutually reinforce each other. To effectively train UniMedVL, we curate UniMedVL-5M, the first large-scale medical dataset comprising over 5.6M instances across 8 medical imaging modalities, tailored for multimodal input-output tasks in unified medical understanding and generation. Experimental results demonstrate that UniMedVL achieves competitive performance on five medical understanding benchmarks. Crucially, UniMedVL natively supports diverse interleaved generation tasks, e.g., virtual staining, super-resolution, cross-modal synthesis, essential for complex medical workflows. Our code and dataset are publicly available.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 86 of its 116 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

86 of 116 references.

References · 86

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  5. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  6. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  8. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  10. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  12. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
38 more
  1. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  2. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  3. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  4. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  5. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  6. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  7. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  8. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  9. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  10. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  11. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  12. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  13. 2025MedGemma Technical Reportcited by 398
  14. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  15. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  16. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  17. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  18. 2018Diverse Image-to-Image Translation via Disentangled Representationscited by 296
  19. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  20. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
  21. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  22. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 194
  23. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 192
  24. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 182
  25. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  26. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
  27. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
  28. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  29. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 112
  30. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  31. 2024CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formatscited by 100
  32. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  33. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
  34. 2020MedICaT: A Dataset of Medical Images, Captions, and Textual Referencescited by 71
  35. 2022Integrated multimodal artificial intelligence framework for healthcare applicationscited by 69
  36. 2022BCI: Breast Cancer Immunohistochemical Image Generation through Pyramid Pix2pixcited by 61
  37. 2023XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Modelscited by 61
  38. 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understandingcited by 54

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.