mlsanityData to 7 Oct 2026

From Understanding to Engagement: Personalized pharmacy Video Clips via Vision Language Models (VLMs)

arXiv 2601.05059 ↗First posted 8 Jan 2026cs.CV cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision Language Models (VLMs) are poised to revolutionize the digital transformation of pharmacyceutical industry by enabling intelligent, scalable, and automated multi-modality content processing. Traditional manual annotation of heterogeneous data modalities (text, images, video, audio, and web links), is prone to inconsistencies, quality degradation, and inefficiencies in content utilization. The sheer volume of long video and audio data further exacerbates these challenges, (e.g. long clinical trial interviews and educational seminars). Here, we introduce a domain adapted Video to Video Clip Generation framework that integrates Audio Language Models (ALMs) and Vision Language Models (VLMs) to produce highlight clips. Our contributions are threefold: (i) a reproducible Cut & Merge algorithm with fade in/out and timestamp normalization, ensuring smooth transitions and audio/visual alignment; (ii) a personalization mechanism based on role definition and prompt injection for tailored outputs (marketing, training, regulatory); (iii) a cost efficient e2e pipeline strategy balancing ALM/VLM enhanced processing. Evaluations on Video MME benchmark (900) and our proprietary dataset of 16,159 pharmacy videos across 14 disease areas demonstrate 3 to 4 times speedup, 4 times cost reduction, and competitive clip quality. Beyond efficiency gains, we also report our methods improved clip coherence scores (0.348) and informativeness scores (0.721) over state of the art VLM baselines (e.g., Gemini 2.5 Pro), highlighting the potential of transparent, custom extractive, and compliance supporting video summarization for life sciences.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 45 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

16 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Position: Restructuring of Categories and Implementation of Guidelines Essential for VLM Adoption in Healthcare12 May
  2. 2025OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding20 Apr
  3. 2024OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining23 Nov

Other references · 13

  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  4. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  5. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  6. 2018Video-to-Video Synthesiscited by 388
  7. 2016Generating Videos with Scene Dynamicscited by 376
  8. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
  9. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 45
  10. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  11. 2025LongDiff: Training-Free Long Video Generation in One Gocited by 13
  12. 2020Ultrasound Video Summarization using Deep Reinforcement Learningcited by 10
1 more
  1. 2020Unsupervised Multimodal Video-to-Video Translation via Self-Supervised Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Are Two LLMs Better Than One? A Student-Teacher Dual-Head LLMs Architecture for Pharmaceutical Content Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Vision Language Models for Pharmaceutical Long Form Video Reasoning on Industrial GenAI Platform
  2. 2025Fleming-VL: Towards Universal Medical Visual Reasoning with Multimodal LLMs
  3. 2025AdCare-VLM: Towards a Unified and Pre-aligned Latent Representation for Healthcare Video Understanding
  4. 2024VividMed: Vision Language Model with Versatile Visual Grounding for Medicine
  5. 2025How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?
  6. 2025How Far Have Medical Vision-Language Models Come? A Comprehensive Benchmarking Study
  7. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review
  8. 2025Vision Language Models in Medicine