mlsanityData to 7 Oct 2026

Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology

arXiv 2605.03352 ↗First posted 5 May 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated robust capabilities in recognizing everyday human activities, yet their potential for analyzing clinically significant involuntary movements in neurological disorders remains largely unexplored. This pilot study evaluates the capability of MLLMs for automated recognition of pathological movements in seizure videos. We assessed the zero-shot performance of state-of-the-art MLLMs on 20 ILAE-defined semiological features across 90 clinical seizure recordings. MLLMs outperformed fine-tuned Convolutional Neural Network (CNN) and Vision Transformer (ViT) baseline models on 13 of 18 features without task-specific training, demonstrating particular strength in recognizing salient postural and contextual features while struggling with subtle, high-frequency movements. Feature-targeted signal enhancement (facial cropping, pose estimation, audio denoising) improved performance on 10 of 20 features. Expert evaluation showed that 94.3 percent of MLLM-generated explanations for correctly predicted cases achieved at least 60 percent faithfulness scores, aligning with epileptologist reasoning. These findings demonstrate the potential of adapting general-purpose MLLMs for specialized clinical video analysis through targeted preprocessing strategies, offering a path toward interpretable, efficient diagnostic assistance. Our code is publicly available at https://github.com/LinaZhangUCLA/PathMotionMLLM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 34 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

21 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation6 Dec

Other references · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  9. 2018SlowFast Networks for Video Recognitioncited by 1,862
  10. 2021ViViT: A Video Vision Transformercited by 1,610
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  12. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
8 more
  1. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  2. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  3. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  4. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  5. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
  6. 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 165
  7. 2023MedViT: A Robust Vision Transformer for Generalized Medical Image Classificationcited by 89
  8. 2025MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Modelscited by 69

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding
  2. 2026NeuroVLM-Bench: Evaluation of Vision-Enabled Large Language Models for Clinical Reasoning in Neurological Disorders
  3. 2026A Neurosymbolic Framework for Interpretable Skeleton-Based Seizure Detection via Concept-Driven Logical Reasoning
  4. 2024SemioLLM: Evaluating Large Language Models for Diagnostic Reasoning from Unstructured Clinical Narratives in Epilepsy
  5. 2025Brain-Adapter: Enhancing Neurological Disorder Analysis with Adapter-Tuning Multimodal Large Language Models
  6. 2026Multimodal Pretraining for Generalizable EEG Representation Learning
  7. 2026RAICL: Retrieval-Augmented In-Context Learning for Vision-Language-Model Based EEG Seizure Detection
  8. 2026PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?