# Going Deeper than Tracking: a Survey of Computer-Vision Based Recognition of Animal Pain and Affective States

arXiv: https://arxiv.org/abs/2206.08405 · HTML: https://mlsanity.com/paper/2206.08405/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 140 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 26 of 26)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax](https://mlsanity.com/paper/2006.10408/) (2020)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020)
- [Facial Landmark Detection: a Literature Survey](https://mlsanity.com/paper/1805.05563/) (2018)
- [Survey on Emotional Body Gesture Recognition](https://mlsanity.com/paper/1801.07481/) (2018)
- [A Deeper Look at Facial Expression Dataset Bias](https://mlsanity.com/paper/1904.11150/) (2019)
- [Facial Feature Point Detection: A Comprehensive Survey](https://mlsanity.com/paper/1410.1037/) (2014)
- [Foundations of data imbalance and solutions for a data democracy](https://mlsanity.com/paper/2108.00071/) (2021)
- [Dynamics are Important for the Recognition of Equine Pain in Video](https://mlsanity.com/paper/1901.02106/) (2019)
- [Automated Detection of Equine Facial Action Units](https://mlsanity.com/paper/2102.08983/) (2021)
- [Sharing Pain: Using Pain Domain Transfer for Video Recognition of Low Grade Orthopedic Pain in Horses](https://mlsanity.com/paper/2105.10313/) (2021)
- [Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation](https://mlsanity.com/paper/2108.13258/) (2021)
- [Emotion Recognition in Horses with Convolutional Neural Networks](https://mlsanity.com/paper/2105.11953/) (2021)

## Built on by (most cited first, 2 of 2)

- [Video-based estimation of pain indicators in dogs](https://mlsanity.com/paper/2209.13296/) (2022)
- [Semantic Style Transfer for Enhancing Animal Facial Landmark Detection](https://mlsanity.com/paper/2505.05640/) (2025)

## Abstract (arXiv)

Advances in animal motion tracking and pose recognition have been a game changer in the study of animal behavior. Recently, an increasing number of works go 'deeper' than tracking, and address automated recognition of animals' internal states such as emotions and pain with the aim of improving animal welfare, making this a timely moment for a systematization of the field. This paper provides a comprehensive survey of computer vision-based research on recognition of affective states and pain in animals, addressing both facial and bodily behavior analysis. We summarize the efforts that have been presented so far within this topic -- classifying them across different dimensions, highlight challenges and research gaps, and provide best practice recommendations for advancing the field, and some future directions for research.