mlsanityData to 7 Oct 2026

Open-Source Tools for Behavioral Video Analysis: Setup, Methods, and Development

arXiv 2204.02842 ↗First posted 6 Apr 2022q-bio.QM cs.CV q-bio.NCcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently developed methods for video analysis, especially models for pose estimation and behavior classification, are transforming behavioral quantification to be more precise, scalable, and reproducible in fields such as neuroscience and ethology. These tools overcome long-standing limitations of manual scoring of video frames and traditional "center of mass" tracking algorithms to enable video analysis at scale. The expansion of open-source tools for video acquisition and analysis has led to new experimental approaches to understand behavior. Here, we review currently available open-source tools for video analysis and discuss how to set up these methods for labs new to video recording. We also discuss best practices for developing and using video analysis methods, including community-wide standards and critical needs for the open sharing of datasets and code, more widespread comparisons of video analysis methods, and better documentation for these methods especially for new users. We encourage broader adoption and continued development of these tools, which have tremendous potential for accelerating scientific progress in understanding the brain and behavior.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 108 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

19 of 108 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2018Datasheets for Datasetscited by 1,371
  6. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  7. 2019An Introduction to Variational Autoencoderscited by 997
  8. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  9. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  10. 2018Markerless tracking of user-defined features with deep learningcited by 203
  11. 2019The VIA Annotation Software for Images, Audio and Videocited by 166
  12. 2016DeepCoder: Learning to Write Programscited by 145
7 more
  1. 2020Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructurecited by 138
  2. 2020Task Programming: Learning Data Efficient Behavior Representationscited by 27
  3. 2021Self-Supervised Keypoint Discovery in Behavioral Videoscited by 23
  4. 2013Mapping the stereotyped behaviour of freely-moving fruit fliescited by 21
  5. 2017Measuring behavior across scalescited by 10
  6. 2021Unsupervised Learning of Neurosymbolic Encoderscited by 8
  7. 2016SimBa: An Efficient Tool for Approximating Rips-filtration Persistence via Simplicial Batch-collapsecited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Computer Vision for Primate Behavior Analysis in the Wildcited by 10
  2. 2024Learning Keypoints for Multi-Agent Behavior Analysis using Self-Supervisioncited by 2
  3. 2025Mouse Lockbox Dataset: Behavior Recognition for Mice Solving Lockboxescited by 0
  4. 2025How many classes do we need to see for novel class discovery?cited by 0