mlsanityData to 14 Oct 2026

Preliminary Use of Vision Language Model Driven Extraction of Mouse Behavior Towards Understanding Fear Expression

arXiv 2510.19160 ↗First posted 22 Oct 2025cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Integration of diverse data will be a pivotal step towards improving scientific explorations in many disciplines. This work establishes a vision-language model (VLM) that encodes videos with text input in order to classify various behaviors of a mouse existing in and engaging with their environment. Importantly, this model produces a behavioral vector over time for each subject and for each session the subject undergoes. The output is a valuable dataset that few programs are able to produce with as high accuracy and with minimal user input. Specifically, we use the open-source Qwen2.5-VL model and enhance its performance through prompts, in-context learning (ICL) with labeled examples, and frame-level preprocessing. We found that each of these methods contributes to improved classification, and that combining them results in strong F1 scores across all behaviors, including rare classes like freezing and fleeing, without any model fine-tuning. Overall, this model will support interdisciplinary researchers studying mouse behavior by enabling them to integrate diverse behavioral features, measured across multiple time points and environments, into a comprehensive dataset that can address complex research questions.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 26 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

10 of 26 references.

References · 10

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  5. 2023Meta-in-context learning in large language modelscited by 59
  6. 2024In-context learning enables multimodal large language models to classify cancer pathology imagescited by 38
  7. 2025Improving LLM Video Understanding with 16 Frames Per Secondcited by 26
  8. 2024VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understandingcited by 17
  9. 2025Can Multimodal LLMs do Visual Temporal Understanding and Reasoning? The answer is No!cited by 15
  10. 2022Characterizing the structure of mouse behavior using Motion Sequencingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Transforming Behavioral Neuroscience Discovery with In-Context Learning and AI-Enhanced Tensor Methodscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MouseGPT: A Large-scale Vision-Language Model for Mouse Behavior Analysis
  2. 2026Rodent-Bench
  3. 2026BehaviorVLM: Unified Finetuning-Free Behavioral Understanding with Vision-Language Reasoning
  4. 2023AmadeusGPT: a natural language interface for interactive animal behavioral analysis
  5. 2024GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding
  6. 2024ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition
  7. 2026From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model
  8. 2026MTT-Bench: Predicting Social Dominance in Mice via Multimodal Large Language Models