# MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understanding

arXiv: https://arxiv.org/abs/2306.00576 · HTML: https://mlsanity.com/paper/2306.00576/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 52 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding](https://mlsanity.com/paper/2204.08129/) (2022)
- [CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning](https://mlsanity.com/paper/2103.16392/) (2021)
- [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019)
- [Pretraining boosts out-of-domain robustness for pose estimation](https://mlsanity.com/paper/1909.11229/) (2019)
- [Deep learning tools for the measurement of animal behavior in neuroscience](https://mlsanity.com/paper/1909.13868/) (2019)
- [AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces](https://mlsanity.com/paper/1909.04951/) (2019)
- [Interspecies Knowledge Transfer for Facial Keypoint Detection](https://mlsanity.com/paper/1704.04023/) (2017)
- [Exploring Fine-Grained Audiovisual Categorization with the SSW60 Dataset](https://mlsanity.com/paper/2207.10664/) (2022)
- [Human and Sheep Facial Landmarks Localisation by Triplet Interpolated Features](https://mlsanity.com/paper/1509.04954/) (2015)
- [SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation](https://mlsanity.com/paper/2108.00249/) (2021)
- [Online Collective Animal Movement Activity Recognition](https://mlsanity.com/paper/1811.09067/) (2018)

## Built on by (most cited first, 30 of 36)

- [OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining](https://mlsanity.com/paper/2411.15421/) (2024)
- [PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition](https://mlsanity.com/paper/2401.13554/) (2024)
- [BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos](https://mlsanity.com/paper/2405.17698/) (2024)
- [BuckTales : A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes](https://mlsanity.com/paper/2411.06896/) (2024)
- [MammAlps: A multi-view video behavior monitoring dataset of wild mammals in the Swiss Alps](https://mlsanity.com/paper/2503.18223/) (2025)
- [Smoke and Mirrors in Causal Downstream Tasks](https://mlsanity.com/paper/2405.17151/) (2024)
- [Computer Vision for Primate Behavior Analysis in the Wild](https://mlsanity.com/paper/2401.16424/) (2024)
- [The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification](https://mlsanity.com/paper/2511.15622/) (2025)
- [MOVE: Motion-Guided Few-Shot Video Object Segmentation](https://mlsanity.com/paper/2507.22061/) (2025)
- [The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition](https://mlsanity.com/paper/2502.21201/) (2025)
- [Forecasting Motion in the Wild](https://mlsanity.com/paper/2604.01015/) (2026)
- [Animal Behavior Analysis Methods Using Deep Learning: A Survey](https://mlsanity.com/paper/2405.14002/) (2024)
- [OpenAnimalTracks: A Dataset for Animal Track Recognition](https://mlsanity.com/paper/2406.09647/) (2024)
- [Global and Local Entailment Learning for Natural World Imagery](https://mlsanity.com/paper/2506.21476/) (2025)
- [Reinforcement Learning from Wild Animal Videos](https://mlsanity.com/paper/2412.04273/) (2024)
- [PriVi: Towards A General-Purpose Video Model For Primate Behavior In The Wild](https://mlsanity.com/paper/2511.09675/) (2025)
- [Towards Application-Specific Evaluation of Vision Models: Case Studies in Ecology and Biology](https://mlsanity.com/paper/2505.02825/) (2025)
- [Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos](https://mlsanity.com/paper/2505.05681/) (2025)
- [Grounding Intelligence in Movement](https://mlsanity.com/paper/2507.02771/) (2025)
- [AnimalClue: Recognizing Animals by their Traces](https://mlsanity.com/paper/2507.20240/) (2025)
- [VC-Agent: An Interactive Agent for Customized Video Dataset Collection](https://mlsanity.com/paper/2509.21291/) (2025)
- [BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representations](https://mlsanity.com/paper/2602.19874/) (2026)
- [WildDepth: A Multimodal Dataset for 3D Wildlife Perception and Depth Estimation](https://mlsanity.com/paper/2603.16816/) (2026)
- [Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data](https://mlsanity.com/paper/2607.09876/) (2026)
- [Kirin: Animal Motion Generation from In-the-Wild Video](https://mlsanity.com/paper/2609.01823/) (2026)
- [From Forest to Zoo: Great Ape Behavior Recognition with ChimpBehave](https://mlsanity.com/paper/2405.20025/) (2024)
- [A Review on Coarse to Fine-Grained Animal Action Recognition](https://mlsanity.com/paper/2506.01214/) (2025)
- [BioBench: A Blueprint to Move Beyond ImageNet for Scientific ML Benchmarks](https://mlsanity.com/paper/2511.16315/) (2025)
- [Interaction-via-Actions: Cattle Interaction Detection with Joint Learning of Action-Interaction Latent Space](https://mlsanity.com/paper/2512.16133/) (2025)
- [A Universal Action Space for General Behavior Analysis](https://mlsanity.com/paper/2602.09518/) (2026)

## Abstract (arXiv)

Monitoring animal behavior can facilitate conservation efforts by providing key insights into wildlife health, population status, and ecosystem function. Automatic recognition of animals and their behaviors is critical for capitalizing on the large unlabeled datasets generated by modern video devices and for accelerating monitoring efforts at scale. However, the development of automated recognition systems is currently hindered by a lack of appropriately labeled datasets. Existing video datasets 1) do not classify animals according to established biological taxonomies; 2) are too small to facilitate large-scale behavioral studies and are often limited to a single species; and 3) do not feature temporally localized annotations and therefore do not facilitate localization of targeted behaviors within longer video sequences. Thus, we propose MammalNet, a new large-scale animal behavior dataset with taxonomy-guided annotations of mammals and their common behaviors. MammalNet contains over 18K videos totaling 539 hours, which is ~10 times larger than the largest existing animal behavior dataset. It covers 17 orders, 69 families, and 173 mammal categories for animal categorization and captures 12 high-level animal behaviors that received focus in previous animal behavior studies. We establish three benchmarks on MammalNet: standard animal and behavior recognition, compositional low-shot animal and behavior recognition, and behavior detection. Our dataset and code have been made available at: https://mammal-net.github.io.