# RGBD-Dog: Predicting Canine Pose from RGBD Sensors

arXiv: https://arxiv.org/abs/2004.07788 · HTML: https://mlsanity.com/paper/2004.07788/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2020.

## Built on by (most cited first, 30 of 35)

- [BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information](https://mlsanity.com/paper/2203.15536/) (2022)
- [Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects](https://mlsanity.com/paper/2112.11347/) (2021)
- [Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos](https://mlsanity.com/paper/2403.17103/) (2024)
- [AniMer: Animal Pose and Shape Estimation Using Family Aware Transformer](https://mlsanity.com/paper/2412.00837/) (2024)
- [hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern Recognition](https://mlsanity.com/paper/2106.10102/) (2021)
- [3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture](https://mlsanity.com/paper/2303.13174/) (2023)
- [Coarse-to-fine Animal Pose and Shape Estimation](https://mlsanity.com/paper/2111.08176/) (2021)
- [T-LEAP: Occlusion-robust pose estimation of walking cows using temporal information](https://mlsanity.com/paper/2104.08029/) (2021)
- [OpenApePose: a database of annotated ape photographs for pose estimation](https://mlsanity.com/paper/2212.00741/) (2022)
- [SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation](https://mlsanity.com/paper/2108.00249/) (2021)
- [Web-Scale Collection of Video Data for 4D Animal Reconstruction](https://mlsanity.com/paper/2511.01169/) (2025)
- [Object Wake-up: 3D Object Rigging from a Single Image](https://mlsanity.com/paper/2108.02708/) (2021)
- [Artemis: Articulated Neural Pets with Appearance and Motion synthesis](https://mlsanity.com/paper/2202.05628/) (2022)
- [Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images](https://mlsanity.com/paper/2410.03438/) (2024)
- [AniMer+: Unified Pose and Shape Estimation Across Mammalia and Aves via Family-Aware Transformer](https://mlsanity.com/paper/2508.00298/) (2025)
- [4D-Animal: Freely Reconstructing Animatable 3D Animals from Videos](https://mlsanity.com/paper/2507.10437/) (2025)
- [Imitation and Adaptation Based on Consistency: A Quadruped Robot Imitates Animals from Videos Using Deep Reinforcement Learning](https://mlsanity.com/paper/2203.05973/) (2022)
- [DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recovery](https://mlsanity.com/paper/2510.24117/) (2025)
- [NeuralFur: Animal Fur Reconstruction From Multi-View Images](https://mlsanity.com/paper/2601.12481/) (2026)
- [Self-supervised Secondary Landmark Detection via 3D Representation Learning](https://mlsanity.com/paper/2110.00543/) (2021)
- [DynaDog+T: A Parametric Animal Model for Synthetic Canine Image Generation](https://mlsanity.com/paper/2107.07330/) (2021)
- [GTT-Net: Learned Generalized Trajectory Triangulation](https://mlsanity.com/paper/2109.03408/) (2021)
- [High Fidelity 3D Reconstructions with Limited Physical Views](https://mlsanity.com/paper/2110.11599/) (2021)
- [Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis](https://mlsanity.com/paper/2410.09312/) (2024)
- [Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals](https://mlsanity.com/paper/2508.16062/) (2025)
- [MBW: Multi-view Bootstrapping in the Wild](https://mlsanity.com/paper/2210.01721/) (2022)
- [Of Mice and Pose: 2D Mouse Pose Estimation from Unlabelled Data and Synthetic Prior](https://mlsanity.com/paper/2307.13361/) (2023)
- [Benchmarking Monocular 3D Dog Pose Estimation Using In-The-Wild Motion Capture Data](https://mlsanity.com/paper/2406.14412/) (2024)
- [CLHOP: Combined Audio-Video Learning for Horse 3D Pose and Shape Estimation](https://mlsanity.com/paper/2407.01244/) (2024)
- [BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representations](https://mlsanity.com/paper/2602.19874/) (2026)

## Abstract (arXiv)

The automatic extraction of animal reb3D pose from images without markers is of interest in a range of scientific fields. Most work to date predicts animal pose from RGB images, based on 2D labelling of joint positions. However, due to the difficult nature of obtaining training data, no ground truth dataset of 3D animal motion is available to quantitatively evaluate these approaches. In addition, a lack of 3D animal pose data also makes it difficult to train 3D pose-prediction methods in a similar manner to the popular field of body-pose prediction. In our work, we focus on the problem of 3D canine pose estimation from RGBD images, recording a diverse range of dog breeds with several Microsoft Kinect v2s, simultaneously obtaining the 3D ground truth skeleton via a motion capture system. We generate a dataset of synthetic RGBD images from this data. A stacked hourglass network is trained to predict 3D joint locations, which is then constrained using prior models of shape and pose. We evaluate our model on both synthetic and real RGBD images and compare our results to previously published work fitting canine models to images. Finally, despite our training set consisting only of dog data, visual inspection implies that our network can produce good predictions for images of other quadrupeds -- e.g. horses or cats -- when their pose is similar to that contained in our training set.