Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Deciphering animal intent is a fundamental challenge in computational ethology, largely because of semantic aliasing, the phenomenon where identical external signals (e.g., a cat's purr) correspond to radically different internal states depending on physiological context. Existing Multimodal Large Language Models (MLLMs) are blind to high-frequency biological time-series data, restricting them to superficial behavioural pattern matching rather than genuine latent-state reasoning. To bridge this gap, we introduce Meow-Omni 1, the first open-source, quad-modal MLLM purpose-built for computational ethology. It natively fuses video, audio, and physiological time-series streams with textual reasoning. Through targeted architectural adaptation, we integrate specialized scientific encoders into a unified backbone and formalize intent inference via physiologically grounded cross-modal alignment. Evaluated on MeowBench, a novel, expert-verified quad-modal benchmark, Meow-Omni 1 achieves state-of-the-art intent-recognition accuracy (71.16%), substantially outperforming leading vision-language and omni-modal baselines. We release the complete open-source pipeline including model weights, training framework, and the Meow-10K dataset, to establish a scalable paradigm for inter-species intent understanding and to advance foundation models toward real-world veterinary diagnostics and wildlife conservation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 47 references that are in the map. First posted on arXiv 9 May 2026.
Builds on
12 of 47 references.
References · 12
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
- 2021AST: Audio Spectrogram Transformercited by 643
- 2025Qwen3-Omni Technical Reportcited by 497
- 2022AVES: Animal Vocalization Encoder based on Self-Supervisioncited by 53
- 2022SuperAnimal pretrained pose estimation models for behavioral analysiscited by 39
- 2021Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcastscited by 30
- 2025Can Masked Autoencoders Also Listen to Birds?cited by 26
- 2021Animal behavior classification via deep learning on embedded systemscited by 9
- 2025Perch 2.0 transfers 'whale' to underwater taskscited by 7
- 2026Action-Guided Attention for Video Action Anticipationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding
- 2023AmadeusGPT: a natural language interface for interactive animal behavioral analysis
- 2026BehaviorVLM: Unified Finetuning-Free Behavioral Understanding with Vision-Language Reasoning
- 2026K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos
- 2024animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics
- 2026MAviS: A Multimodal Conversational Assistant For Avian Species
- 2026Rodent-Bench
- 2025MouseGPT: A Large-scale Vision-Language Model for Mouse Behavior Analysis