mlsanityData to 7 Oct 2026

A Vision-Language Model for Focal Liver Lesion Classification

arXiv 2505.03350 ↗First posted 6 May 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Accurate classification of focal liver lesions is crucial for diagnosis and treatment in hepatology. However, traditional supervised deep learning models depend on large-scale annotated datasets, which are often limited in medical imaging. Recently, Vision-Language models (VLMs) such as Contrastive Language-Image Pre-training model (CLIP) has been applied to image classifications. Compared to the conventional convolutional neural network (CNN), which classifiers image based on visual information only, VLM leverages multimodal learning with text and images, allowing it to learn effectively even with a limited amount of labeled data. Inspired by CLIP, we pro-pose a Liver-VLM, a model specifically designed for focal liver lesions (FLLs) classification. First, Liver-VLM incorporates class information into the text encoder without introducing additional inference overhead. Second, by calculating the pairwise cosine similarities between image and text embeddings and optimizing the model with a cross-entropy loss, Liver-VLM ef-fectively aligns image features with class-level text features. Experimental results on MPCT-FLLs dataset demonstrate that the Liver-VLM model out-performs both the standard CLIP and MedCLIP models in terms of accuracy and Area Under the Curve (AUC). Further analysis shows that using a lightweight ResNet18 backbone enhances classification performance, particularly under data-constrained conditions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 16 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

9 of 16 references.

References · 9

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  7. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  8. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  9. 2021PA-ResSeg: A Phase Attention Residual Network for Liver Tumor Segmentation from Multi-phase CT Imagescited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study
  2. 2026Does medical specialization of VLMs enhance discriminative power?: A comprehensive investigation through feature distribution analysis
  3. 2026Hepato-LLaVA: An Expert MLLM with Sparse Topo-Pack Attention for Hepatocellular Pathology Analysis on Whole Slide Images
  4. 2025Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context
  5. 2024Boosting Vision-Language Models for Histopathology Classification: Predict all at once
  6. 2025Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
  7. 2025Medical Knowledge Intervention Prompt Tuning for Medical Image Classification
  8. 2025AutoRad-Lung: A Radiomic-Guided Prompting Autoregressive Vision-Language Model for Lung Nodule Malignancy Prediction