# The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification

arXiv: https://arxiv.org/abs/2511.15622 · HTML: https://mlsanity.com/paper/2511.15622/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 30 of 39)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [TAO: A Large-Scale Benchmark for Tracking Any Object](https://mlsanity.com/paper/2005.10356/) (2020)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding](https://mlsanity.com/paper/2204.08129/) (2022)
- [Efficient Pipeline for Camera Trap Image Review](https://mlsanity.com/paper/1907.06772/) (2019)
- [General Object Foundation Model for Images and Videos at Scale](https://mlsanity.com/paper/2312.09158/) (2023)
- [BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video](https://mlsanity.com/paper/2209.12118/) (2022)
- [LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models](https://mlsanity.com/paper/2501.18954/) (2025)
- [APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking](https://mlsanity.com/paper/2206.05683/) (2022)
- [Tracking Every Thing in the Wild](https://mlsanity.com/paper/2207.12978/) (2022)
- [AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wild](https://mlsanity.com/paper/2205.00158/) (2022)
- [MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understanding](https://mlsanity.com/paper/2306.00576/) (2023)
- [GMOT-40: A Benchmark for Generic Multiple Object Tracking](https://mlsanity.com/paper/2011.11858/) (2020)
- [3D-ZeF: A 3D Zebrafish Tracking Benchmark Dataset](https://mlsanity.com/paper/2006.08466/) (2020)
- [ChimpACT: A Longitudinal Dataset for Understanding Chimpanzee Behaviors](https://mlsanity.com/paper/2310.16447/) (2023)
- [PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition](https://mlsanity.com/paper/2401.13554/) (2024)
- [3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture](https://mlsanity.com/paper/2303.13174/) (2023)
- [Visual Recognition of Great Ape Behaviours in the Wild](https://mlsanity.com/paper/2011.10759/) (2020)
- [BoostTrack++: using tracklet information to detect more objects in multiple object tracking](https://mlsanity.com/paper/2408.13003/) (2024)
- [Triple-stream Deep Metric Learning of Great Ape Behavioural Actions](https://mlsanity.com/paper/2301.02642/) (2023)
- [BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos](https://mlsanity.com/paper/2405.17698/) (2024)
- [BuckTales : A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes](https://mlsanity.com/paper/2411.06896/) (2024)

## Built on by (most cited first, 7 of 7)

- [Deep in the Jungle: Towards Automating Chimpanzee Population Estimation](https://mlsanity.com/paper/2601.22917/) (2026)
- [Cross-Modal Corroboration for Annotation-Free Wildlife Monitoring](https://mlsanity.com/paper/2606.21613/) (2026)
- [Automating Visual Recognition of Leprosy in Wild Chimpanzees](https://mlsanity.com/paper/2607.17395/) (2026)
- [The PanAf-SBR Dataset: Social Behaviour Recognition for Wild Great Apes](https://mlsanity.com/paper/2607.17399/) (2026)
- [Teach a Molmo2Fish: Towards interactive fish tracking with natural language guidance](https://mlsanity.com/paper/2608.18602/) (2026)
- [VastMAT: A Large-Scale Multi-Category Benchmark for Multi-Animal Tracking](https://mlsanity.com/paper/2609.34390/) (2026)
- [When Predicting Nothing Beats SAM 3: Revisiting Evaluation in Video Object Segmentation](https://mlsanity.com/paper/2610.02946/) (2026)

## Abstract (arXiv)

Automated video analysis is critical for wildlife conservation. A foundational task in this domain is multi-animal tracking (MAT), which underpins applications such as individual re-identification and behavior recognition. However, existing datasets are limited in scale, constrained to a few species, or lack sufficient temporal and geographical diversity - leaving no suitable benchmark for training general-purpose MAT models applicable across wild animal populations. To address this, we introduce SA-FARI, the largest open-source MAT dataset for wild animals. It comprises 11,609 camera trap videos collected over approximately 10 years (2014-2024) from 741 locations across 4 continents, spanning 99 species categories. Each video is exhaustively annotated culminating in ~46 hours of densely annotated footage containing 16,224 masklet identities and 942,702 individual bounding boxes, segmentation masks, and species labels. Alongside the task-specific annotations, we publish anonymized camera trap locations for each video. Finally, we present comprehensive benchmarks on SA-FARI using state-of-the-art vision-language models for detection and tracking, including SAM 3, evaluated with both species-specific and generic animal prompts. We also compare against vision-only methods developed specifically for wildlife analysis. SA-FARI is the first large-scale dataset to combine high species diversity, multi-region coverage, and high-quality spatio-temporal annotations, offering a new foundation for advancing generalizable multianimal tracking in the wild. The dataset is available at https://www.conservationxlabs.com/sa-fari.