# Of Mice and Pose: 2D Mouse Pose Estimation from Unlabelled Data and Synthetic Prior

arXiv: https://arxiv.org/abs/2307.13361 · HTML: https://mlsanity.com/paper/2307.13361/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 62 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation](https://mlsanity.com/paper/1611.09326/) (2016)
- [Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"](https://mlsanity.com/paper/1908.07201/) (2019)
- [DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model](https://mlsanity.com/paper/1605.03170/) (2016)
- [Shape and Viewpoint without Keypoints](https://mlsanity.com/paper/2007.10982/) (2020)
- [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020)
- [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019)
- [Flowing ConvNets for Human Pose Estimation in Videos](https://mlsanity.com/paper/1506.02897/) (2015)
- [From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/2103.14843/) (2021)
- [Pretraining boosts out-of-domain robustness for pose estimation](https://mlsanity.com/paper/1909.11229/) (2019)
- [Deep learning tools for the measurement of animal behavior in neuroscience](https://mlsanity.com/paper/1909.13868/) (2019)
- [RGBD-Dog: Predicting Canine Pose from RGBD Sensors](https://mlsanity.com/paper/2004.07788/) (2020)
- [T-LEAP: Occlusion-robust pose estimation of walking cows using temporal information](https://mlsanity.com/paper/2104.08029/) (2021)
- [Articulated motion discovery using pairs of trajectories](https://mlsanity.com/paper/1411.7883/) (2014)
- [Learning 3D Deformation of Animals from 2D Images](https://mlsanity.com/paper/1507.07646/) (2015)
- [SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation](https://mlsanity.com/paper/2108.00249/) (2021)
- [Prior-Aware Synthetic Data to the Rescue: Animal Pose Estimation with Very Limited Real Data](https://mlsanity.com/paper/2208.13944/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Horse with no Labels: Self-Supervised Horse Pose Estimation from Unlabelled Images and Synthetic Prior](https://mlsanity.com/paper/2308.03411/) (2023)

## Abstract (arXiv)

Numerous fields, such as ecology, biology, and neuroscience, use animal recordings to track and measure animal behaviour. Over time, a significant volume of such data has been produced, but some computer vision techniques cannot explore it due to the lack of annotations. To address this, we propose an approach for estimating 2D mouse body pose from unlabelled images using a synthetically generated empirical pose prior. Our proposal is based on a recent self-supervised method for estimating 2D human pose that uses single images and a set of unpaired typical 2D poses within a GAN framework. We adapt this method to the limb structure of the mouse and generate the empirical prior of 2D poses from a synthetic 3D mouse model, thereby avoiding manual annotation. In experiments on a new mouse video dataset, we evaluate the performance of the approach by comparing pose predictions to a manually obtained ground truth. We also compare predictions with those from a supervised state-of-the-art method for animal pose estimation. The latter evaluation indicates promising results despite the lack of paired training data. Finally, qualitative results using a dataset of horse images show the potential of the setting to adapt to other animal species.