# Challenges in Video-Based Infant Action Recognition: A Critical Examination of the State of the Art

arXiv: https://arxiv.org/abs/2311.12300 · HTML: https://mlsanity.com/paper/2311.12300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 17 of 17)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks](https://mlsanity.com/paper/1707.05740/) (2017)
- [Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.02447/) (2016)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [Invariant Representation Learning for Infant Pose Estimation with Small Data](https://mlsanity.com/paper/2010.06100/) (2020)
- [Skeleton-based Action Recognition Using LSTM and CNN](https://mlsanity.com/paper/1707.02356/) (2017)
- [InfAnFace: Bridging the infant-adult domain gap in facial landmark estimation in the wild](https://mlsanity.com/paper/2110.08935/) (2021)
- [Computer Vision to the Rescue: Infant Postural Symmetry Estimation from Incongruent Annotations](https://mlsanity.com/paper/2207.09352/) (2022)
- [Heuristic Weakly Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2105.10996/) (2021)
- [A Video-based End-to-end Pipeline for Non-nutritive Sucking Action Recognition and Segmentation in Young Infants](https://mlsanity.com/paper/2303.16867/) (2023)
- [Target-Specific Action Classification for Automated Assessment of Human Motor Behavior from Video](https://mlsanity.com/paper/1909.09566/) (2019)
- [BabyNet: A Lightweight Network for Infant Reaching Action Recognition in Unconstrained Environments to Support Future Pediatric Rehabilitation Applications](https://mlsanity.com/paper/2208.04950/) (2022)

## Abstract (arXiv)

Automated human action recognition, a burgeoning field within computer vision, boasts diverse applications spanning surveillance, security, human-computer interaction, tele-health, and sports analysis. Precise action recognition in infants serves a multitude of pivotal purposes, encompassing safety monitoring, developmental milestone tracking, early intervention for developmental delays, fostering parent-infant bonds, advancing computer-aided diagnostics, and contributing to the scientific comprehension of child development. This paper delves into the intricacies of infant action recognition, a domain that has remained relatively uncharted despite the accomplishments in adult action recognition. In this study, we introduce a groundbreaking dataset called ``InfActPrimitive'', encompassing five significant infant milestone action categories, and we incorporate specialized preprocessing for infant data. We conducted an extensive comparative analysis employing cutting-edge skeleton-based action recognition models using this dataset. Our findings reveal that, although the PoseC3D model achieves the highest accuracy at approximately 71%, the remaining models struggle to accurately capture the dynamics of infant actions. This highlights a substantial knowledge gap between infant and adult action recognition domains and the urgent need for data-efficient pipeline models.