# Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation

arXiv: https://arxiv.org/abs/2108.13258 · HTML: https://mlsanity.com/paper/2108.13258/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 74 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localization](https://mlsanity.com/paper/1704.04232/) (2017)
- [Unsupervised Learning of Disentangled Representations from Video](https://mlsanity.com/paper/1705.10915/) (2017)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [3D Ken Burns Effect from a Single Image](https://mlsanity.com/paper/1909.05483/) (2019)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"](https://mlsanity.com/paper/1908.07201/) (2019)
- [Weakly Supervised Action Localization by Sparse Temporal Pooling Network](https://mlsanity.com/paper/1712.05080/) (2017)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)
- [Extreme View Synthesis](https://mlsanity.com/paper/1812.04777/) (2018)
- [W-TALC: Weakly-supervised Temporal Activity Localization and Classification](https://mlsanity.com/paper/1807.10418/) (2018)
- [Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation](https://mlsanity.com/paper/1804.01110/) (2018)
- [Neural Face Editing with Intrinsic Image Disentangling](https://mlsanity.com/paper/1704.04131/) (2017)
- [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019)
- [FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery](https://mlsanity.com/paper/1811.11155/) (2018)
- [Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene](https://mlsanity.com/paper/1712.01812/) (2017)
- [In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representations](https://mlsanity.com/paper/1904.03289/) (2019)
- [AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces](https://mlsanity.com/paper/1909.04951/) (2019)
- [Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation](https://mlsanity.com/paper/1903.08839/) (2019)
- [Transferring Dense Pose to Proximal Animal Classes](https://mlsanity.com/paper/2003.00080/) (2020)
- [Neural Scene Decomposition for Multi-Person Motion Capture](https://mlsanity.com/paper/1903.05684/) (2019)
- [Disentangling Factors of Variation by Mixing Them](https://mlsanity.com/paper/1711.07410/) (2017)
- [hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern Recognition](https://mlsanity.com/paper/2106.10102/) (2021)

## Built on by (most cited first, 2 of 2)

- [Going Deeper than Tracking: a Survey of Computer-Vision Based Recognition of Animal Pain and Affective States](https://mlsanity.com/paper/2206.08405/) (2022)
- [Deep Learning Models for Automated Classification of Dog Emotional States from Facial Expressions](https://mlsanity.com/paper/2206.05619/) (2022)

## Abstract (arXiv)

Timely detection of horse pain is important for equine welfare. Horses express pain through their facial and body behavior, but may hide signs of pain from unfamiliar human observers. In addition, collecting visual data with detailed annotation of horse behavior and pain state is both cumbersome and not scalable. Consequently, a pragmatic equine pain classification system would use video of the unobserved horse and weak labels. This paper proposes such a method for equine pain classification by using multi-view surveillance video footage of unobserved horses with induced orthopaedic pain, with temporally sparse video level pain labels. To ensure that pain is learned from horse body language alone, we first train a self-supervised generative model to disentangle horse pose from its appearance and background before using the disentangled horse pose latent representation for pain classification. To make best use of the pain labels, we develop a novel loss that formulates pain classification as a multi-instance learning problem. Our method achieves pain classification accuracy better than human expert performance with 60% accuracy. The learned latent horse pose representation is shown to be viewpoint covariant, and disentangled from horse appearance. Qualitative analysis of pain classified segments shows correspondence between the pain symptoms identified by our model, and equine pain scales used in veterinary practice.