# The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition

arXiv: https://arxiv.org/abs/2502.21201 · HTML: https://mlsanity.com/paper/2502.21201/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning](https://mlsanity.com/paper/1910.04744/) (2019)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition](https://mlsanity.com/paper/1912.05534/) (2019)
- [Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding](https://mlsanity.com/paper/2204.08129/) (2022)
- [A Comprehensive Study of Deep Video Action Recognition](https://mlsanity.com/paper/2012.06567/) (2020)
- [The iWildCam 2021 Competition Dataset](https://mlsanity.com/paper/2105.03494/) (2021)
- [Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning](https://mlsanity.com/paper/2009.05769/) (2020)
- [Do Image Classifiers Generalize Across Time?](https://mlsanity.com/paper/1906.02168/) (2019)
- [A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes](https://mlsanity.com/paper/2201.10766/) (2022)
- [Motion-aware Contrastive Video Representation Learning via Foreground-background Merging](https://mlsanity.com/paper/2109.15130/) (2021)
- [MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understanding](https://mlsanity.com/paper/2306.00576/) (2023)
- [Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foreground](https://mlsanity.com/paper/2211.12883/) (2022)
- [ChimpACT: A Longitudinal Dataset for Understanding Chimpanzee Behaviors](https://mlsanity.com/paper/2310.16447/) (2023)
- [Iterative Human and Automated Identification of Wildlife Images](https://mlsanity.com/paper/2105.02320/) (2021)
- [Use Your Head: Improving Long-Tail Video Recognition](https://mlsanity.com/paper/2304.01143/) (2023)
- [PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition](https://mlsanity.com/paper/2401.13554/) (2024)
- [How can objects help action recognition?](https://mlsanity.com/paper/2306.11726/) (2023)
- [Human Action Recognition without Human](https://mlsanity.com/paper/1608.07876/) (2016)
- [Visual Recognition of Great Ape Behaviours in the Wild](https://mlsanity.com/paper/2011.10759/) (2020)

## Built on by (most cited first, 5 of 5)

- [The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification](https://mlsanity.com/paper/2511.15622/) (2025)
- [TrAction: Action Recognition with Sparse Trajectories](https://mlsanity.com/paper/2606.03490/) (2026)
- [SmartWilds: Multimodal Wildlife Monitoring Dataset](https://mlsanity.com/paper/2509.18894/) (2025)
- [Automating Visual Recognition of Leprosy in Wild Chimpanzees](https://mlsanity.com/paper/2607.17395/) (2026)
- [The PanAf-SBR Dataset: Social Behaviour Recognition for Wild Great Apes](https://mlsanity.com/paper/2607.17399/) (2026)

## Abstract (arXiv)

Computer vision analysis of camera trap video footage is essential for wildlife conservation, as captured behaviours offer some of the earliest indicators of changes in population health. Recently, several high-impact animal behaviour datasets and methods have been introduced to encourage their use; however, the role of behaviour-correlated background information and its significant effect on out-of-distribution generalisation remain unexplored. In response, we present the PanAf-FGBG dataset, featuring 20 hours of wild chimpanzee behaviours, recorded at over 350 individual camera locations. Uniquely, it pairs every video with a chimpanzee (referred to as a foreground video) with a corresponding background video (with no chimpanzee) from the same camera location. We present two views of the dataset: one with overlapping camera locations and one with disjoint locations. This setup enables, for the first time, direct evaluation of in-distribution and out-of-distribution conditions, and for the impact of backgrounds on behaviour recognition models to be quantified. All clips come with rich behavioural annotations and metadata including unique camera IDs and detailed textual scene descriptions. Additionally, we establish several baselines and present a highly effective latent-space normalisation technique that boosts out-of-distribution performance by +5.42% mAP for convolutional and +3.75% mAP for transformer-based models. Finally, we provide an in-depth analysis on the role of backgrounds in out-of-distribution behaviour recognition, including the so far unexplored impact of background durations (i.e., the count of background frames within foreground videos).