# Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape

arXiv: https://arxiv.org/abs/2308.11737 · HTML: https://mlsanity.com/paper/2308.11737/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 30 of 35)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation](https://mlsanity.com/paper/1808.05942/) (2018)
- [Monocular Total Capture: Posing Face, Body, and Hands in the Wild](https://mlsanity.com/paper/1812.01598/) (2018)
- [Probabilistic Modeling for Human Mesh Recovery](https://mlsanity.com/paper/2108.11944/) (2021)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach](https://mlsanity.com/paper/1704.02447/) (2017)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"](https://mlsanity.com/paper/1908.07201/) (2019)
- [DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare](https://mlsanity.com/paper/1910.00116/) (2019)
- [Self-supervised Learning of Motion Capture](https://mlsanity.com/paper/1712.01337/) (2017)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)
- [Creatures great and SMAL: Recovering the shape and motion of animals from video](https://mlsanity.com/paper/1811.05804/) (2018)
- [Articulation-aware Canonical Surface Mapping](https://mlsanity.com/paper/2004.00614/) (2020)
- [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020)
- [LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery](https://mlsanity.com/paper/2207.03434/) (2022)
- [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019)

## Built on by (most cited first, 30 of 44)

- [Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation](https://mlsanity.com/paper/2501.12202/) (2025)
- [Learning the 3D Fauna of the Web](https://mlsanity.com/paper/2401.02400/) (2024)
- [NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priors](https://mlsanity.com/paper/2403.03122/) (2024)
- [OmniMotionGPT: Animal Motion Generation with Limited Data](https://mlsanity.com/paper/2311.18303/) (2023)
- [AniMer: Animal Pose and Shape Estimation Using Family Aware Transformer](https://mlsanity.com/paper/2412.00837/) (2024)
- [Generative Zoo](https://mlsanity.com/paper/2412.08101/) (2024)
- [Web-Scale Collection of Video Data for 4D Animal Reconstruction](https://mlsanity.com/paper/2511.01169/) (2025)
- [3D-LFM: Lifting Foundation Model](https://mlsanity.com/paper/2312.11894/) (2023)
- [Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images](https://mlsanity.com/paper/2410.03438/) (2024)
- [AniMer+: Unified Pose and Shape Estimation Across Mammalia and Aves via Family-Aware Transformer](https://mlsanity.com/paper/2508.00298/) (2025)
- [4D-Animal: Freely Reconstructing Animatable 3D Animals from Videos](https://mlsanity.com/paper/2507.10437/) (2025)
- [SweepNet: Unsupervised Learning Shape Abstraction via Neural Sweepers](https://mlsanity.com/paper/2407.06305/) (2024)
- [ReVision: Refining Video Diffusion with Explicit 3D Motion Modeling](https://mlsanity.com/paper/2504.21855/) (2025)
- [DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recovery](https://mlsanity.com/paper/2510.24117/) (2025)
- [NeuralFur: Animal Fur Reconstruction From Multi-View Images](https://mlsanity.com/paper/2601.12481/) (2026)
- [DiffSurf: A Transformer-based Diffusion Model for Generating and Reconstructing 3D Surfaces in Pose](https://mlsanity.com/paper/2408.14860/) (2024)
- [FMPose3D: monocular 3D pose estimation via flow matching](https://mlsanity.com/paper/2602.05755/) (2026)
- [Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis](https://mlsanity.com/paper/2410.09312/) (2024)
- [Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance](https://mlsanity.com/paper/2505.18342/) (2025)
- [MeshMamba: State Space Models for Articulated 3D Mesh Generation and Reconstruction](https://mlsanity.com/paper/2507.15212/) (2025)
- [Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals](https://mlsanity.com/paper/2508.16062/) (2025)
- [SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild](https://mlsanity.com/paper/2605.07604/) (2026)
- [Benchmarking Monocular 3D Dog Pose Estimation Using In-The-Wild Motion Capture Data](https://mlsanity.com/paper/2406.14412/) (2024)
- [MoReMouse: Monocular Reconstruction of Laboratory Mouse](https://mlsanity.com/paper/2507.04258/) (2025)
- [AnimalClue: Recognizing Animals by their Traces](https://mlsanity.com/paper/2507.20240/) (2025)
- [DiffPose-Animal: A Language-Conditioned Diffusion Framework for Animal Pose Estimation](https://mlsanity.com/paper/2508.08783/) (2025)
- [VibraVerse: A Large-Scale Geometry-Acoustics Alignment Dataset for Physically-Consistent Multimodal Learning](https://mlsanity.com/paper/2511.20422/) (2025)
- [BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representations](https://mlsanity.com/paper/2602.19874/) (2026)
- [Monocular Mesh Recovery and Body Measurement of Female Saanen Goats](https://mlsanity.com/paper/2602.19896/) (2026)
- [WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring](https://mlsanity.com/paper/2604.24718/) (2026)

## Abstract (arXiv)

Accurately estimating the 3D pose and shape is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. However, research in this area is held back by the lack of a comprehensive and diverse dataset with high-quality 3D pose and shape annotations. In this paper, we propose Animal3D, the first comprehensive dataset for mammal animal 3D pose and shape estimation. Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model. All annotations were labeled and checked manually in a multi-stage process to ensure highest quality results. Based on the Animal3D dataset, we benchmark representative shape and pose estimation models at: (1) supervised learning from only the Animal3D data, (2) synthetic to real transfer from synthetically generated images, and (3) fine-tuning human pose and shape estimation models. Our experimental results demonstrate that predicting the 3D shape and pose of animals across species remains a very challenging task, despite significant advances in human pose estimation. Our results further demonstrate that synthetic pre-training is a viable strategy to boost the model performance. Overall, Animal3D opens new directions for facilitating future research in animal 3D pose and shape estimation, and is publicly available.