# SuperAnimal pretrained pose estimation models for behavioral analysis

arXiv: https://arxiv.org/abs/2203.07436 · HTML: https://mlsanity.com/paper/2203.07436/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 92 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [Creatures great and SMAL: Recovering the shape and motion of animals from video](https://mlsanity.com/paper/1811.05804/) (2018)
- [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020)
- [Benchmarking and Error Diagnosis in Multi-Instance Pose Estimation](https://mlsanity.com/paper/1707.05388/) (2017)

## Built on by (most cited first, 30 of 39)

- [AniMer: Animal Pose and Shape Estimation Using Family Aware Transformer](https://mlsanity.com/paper/2412.00837/) (2024)
- [BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos](https://mlsanity.com/paper/2405.17698/) (2024)
- [MammAlps: A multi-view video behavior monitoring dataset of wild mammals in the Swiss Alps](https://mlsanity.com/paper/2503.18223/) (2025)
- [AmadeusGPT: a natural language interface for interactive animal behavioral analysis](https://mlsanity.com/paper/2307.04858/) (2023)
- [Computer Vision for Primate Behavior Analysis in the Wild](https://mlsanity.com/paper/2401.16424/) (2024)
- [AniMer+: Unified Pose and Shape Estimation Across Mammalia and Aves via Family-Aware Transformer](https://mlsanity.com/paper/2508.00298/) (2025)
- [Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddings](https://mlsanity.com/paper/2410.10744/) (2024)
- [SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications](https://mlsanity.com/paper/2507.03578/) (2025)
- [Adaptive Intelligence: leveraging insights from adaptive behavior in animals to build flexible AI systems](https://mlsanity.com/paper/2411.15234/) (2024)
- [Animal behavioral analysis and neural encoding with transformer-based self-supervised pretraining](https://mlsanity.com/paper/2507.09513/) (2025)
- [Forecasting Motion in the Wild](https://mlsanity.com/paper/2604.01015/) (2026)
- [DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recovery](https://mlsanity.com/paper/2510.24117/) (2025)
- [Probabilistic Prompt Distribution Learning for Animal Pose Estimation](https://mlsanity.com/paper/2503.16120/) (2025)
- [Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis](https://mlsanity.com/paper/2410.09312/) (2024)
- [PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection](https://mlsanity.com/paper/2412.11923/) (2024)
- [Animal Pose Labeling Using General-Purpose Point Trackers](https://mlsanity.com/paper/2506.03868/) (2025)
- [UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation](https://mlsanity.com/paper/2507.02713/) (2025)
- [AnimalFormer: Multimodal Vision Framework for Behavior-based Precision Livestock Farming](https://mlsanity.com/paper/2406.09711/) (2024)
- [RatBodyFormer: Rat Body Surface from Keypoints](https://mlsanity.com/paper/2412.09599/) (2024)
- [MBE-ARI: A Multimodal Dataset Mapping Bi-directional Engagement in Animal-Robot Interaction](https://mlsanity.com/paper/2504.08646/) (2025)
- [Grounding Intelligence in Movement](https://mlsanity.com/paper/2507.02771/) (2025)
- [AnimalClue: Recognizing Animals by their Traces](https://mlsanity.com/paper/2507.20240/) (2025)
- [Articulate3D: Zero-Shot Text-Driven 3D Object Posing](https://mlsanity.com/paper/2508.19244/) (2025)
- [WildDepth: A Multimodal Dataset for 3D Wildlife Perception and Depth Estimation](https://mlsanity.com/paper/2603.16816/) (2026)
- [WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring](https://mlsanity.com/paper/2604.24718/) (2026)
- [PRIMA: Boosting Animal Mesh Recovery with Biological Priors and Test-Time Adaptation](https://mlsanity.com/paper/2606.02366/) (2026)
- [SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality](https://mlsanity.com/paper/2409.08083/) (2024)
- [Towards Texture- And Shape-Independent 3D Keypoint Estimation in Birds](https://mlsanity.com/paper/2505.16633/) (2025)
- [SteerPose: Simultaneous Extrinsic Camera Calibration and Matching from Articulation](https://mlsanity.com/paper/2506.01691/) (2025)
- [Towards Continuous Home Cage Monitoring: An Evaluation of Tracking and Identification Strategies for Laboratory Mice](https://mlsanity.com/paper/2507.07929/) (2025)

## Abstract (arXiv)

Quantification of behavior is critical in applications ranging from neuroscience, veterinary medicine and animal conservation efforts. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present a series of technical innovations that enable a new method, collectively called SuperAnimal, to develop unified foundation models that can be used on over 45 species, without additional human labels. Concretely, we introduce a method to unify the keypoint space across differently labeled datasets (via our generalized data converter) and for training these diverse datasets in a manner such that they don't catastrophically forget keypoints given the unbalanced inputs (via our keypoint gradient masking and memory replay approaches). These models show excellent performance across six pose benchmarks. Then, to ensure maximal usability for end-users, we demonstrate how to fine-tune the models on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If the models are fine-tuned, we show SuperAnimal models are 10-100× more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification in mice and gait analysis in horses. Collectively, this presents a data-efficient solution for animal pose estimation.