# Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis

arXiv: https://arxiv.org/abs/2410.09312 · HTML: https://mlsanity.com/paper/2410.09312/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 76 of its 174 references that are in the map.
- First posted on arXiv 12 Oct 2024.

## Builds on (most cited first, 30 of 76)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Deep Learning-Based Human Pose Estimation: A Survey](https://mlsanity.com/paper/2012.13392/) (2020)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild](https://mlsanity.com/paper/2406.09905/) (2024)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"](https://mlsanity.com/paper/1908.07201/) (2019)
- [Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding](https://mlsanity.com/paper/2204.08129/) (2022)
- [DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model](https://mlsanity.com/paper/1605.03170/) (2016)
- [Shape and Viewpoint without Keypoints](https://mlsanity.com/paper/2007.10982/) (2020)

## Built on by (most cited first, 2 of 2)

- [Articulate3D: Zero-Shot Text-Driven 3D Object Posing](https://mlsanity.com/paper/2508.19244/) (2025)
- [Promptable Animal Pose Tracking Across Species](https://mlsanity.com/paper/2608.04995/) (2026)

## Abstract (arXiv)

Animal pose estimation (APE) aims to locate the animal body parts using a diverse array of sensor and modality inputs (e.g. RGB cameras, LiDAR, infrared, IMU, acoustic and language cues), which is crucial for research across neuroscience, biomechanics, and veterinary medicine. By evaluating 176 papers since 2011, APE methods are categorised by their input sensor and modality types, output forms, learning paradigms, experimental setup, and application domains, presenting detailed analyses of current trends, challenges, and future directions in single- and multi-modality APE systems. The analysis also highlights the transition between human and animal pose estimation, and how innovations in APE can reciprocally enrich human pose estimation and the broader machine learning paradigm. Additionally, 2D and 3D APE datasets and evaluation metrics based on different sensors and modalities are provided. A regularly updated project page is provided here: https://github.com/ChennyDeng/MM-APE.