mlsanityData to 14 Oct 2026

AnimalFormer: Multimodal Vision Framework for Behavior-based Precision Livestock Farming

arXiv 2406.09711 ↗First posted 14 Jun 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce a multimodal vision framework for precision livestock farming, harnessing the power of GroundingDINO, HQSAM, and ViTPose models. This integrated suite enables comprehensive behavioral analytics from video data without invasive animal tagging. GroundingDINO generates accurate bounding boxes around livestock, while HQSAM segments individual animals within these boxes. ViTPose estimates key body points, facilitating posture and movement analysis. Demonstrated on a sheep dataset with grazing, running, sitting, standing, and walking activities, our framework extracts invaluable insights: activity and grazing patterns, interaction dynamics, and detailed postural evaluations. Applicable across species and video resolutions, this framework revolutionizes non-invasive livestock monitoring for activity detection, counting, health assessments, and posture analyses. It empowers data-driven farm management, optimizing animal welfare and productivity through AI-powered behavioral understanding.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 35 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

15 of 35 references.

References · 15

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  10. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  11. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  12. 2021AP-10K: A Benchmark for Animal Pose Estimation in the Wildcited by 123
3 more
  1. 2022APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Trackingcited by 56
  2. 2022SuperAnimal pretrained pose estimation models for behavioral analysiscited by 39
  3. 2023Automated Behavioral Analysis Using Instance Segmentationcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025AI in Agriculture: A Survey of Deep Learning Techniques for Crops, Fisheries and Livestockcited by 0