# Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbanging

arXiv: https://arxiv.org/abs/2101.03478 · HTML: https://mlsanity.com/paper/2101.03478/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 76 references that are in the map.
- First posted on arXiv 10 Jan 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems](https://mlsanity.com/paper/2002.06581/) (2020)
- [Action Recognition: From Static Datasets to Moving Robots](https://mlsanity.com/paper/1701.04925/) (2017)
- [A Wearable Social Interaction Aid for Children with Autism](https://mlsanity.com/paper/2004.14281/) (2020)

## Built on by (most cited first, 15 of 15)

- [Classification of Abnormal Hand Movement for Aiding in Autism Detection: Machine Learning Study](https://mlsanity.com/paper/2108.07917/) (2021)
- [Vision-Based Activity Recognition in Children with Autism-Related Behaviors](https://mlsanity.com/paper/2208.04206/) (2022)
- [MITFAS: Mutual Information based Temporal Feature Alignment and Sampling for Aerial Video Action Recognition](https://mlsanity.com/paper/2303.02575/) (2023)
- [Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images](https://mlsanity.com/paper/2201.11197/) (2022)
- [Computer Vision Estimation of Emotion Reaction Intensity in the Wild](https://mlsanity.com/paper/2303.10741/) (2023)
- [Video-Based Autism Detection with Deep Learning](https://mlsanity.com/paper/2402.16774/) (2024)
- [Classifying Autism from Crowdsourced Semi-Structured Speech Recordings: A Machine Learning Approach](https://mlsanity.com/paper/2201.00927/) (2022)
- [Introducing SSBD+ Dataset with a Convolutional Pipeline for detecting Self-Stimulatory Behaviours in Children using raw videos](https://mlsanity.com/paper/2311.15072/) (2023)
- [An Exploration of Active Learning for Affective Digital Phenotyping](https://mlsanity.com/paper/2204.01915/) (2022)
- [Personalization of Affective Models to Enable Neuropsychiatric Digital Precision Health Interventions: A Feasibility Study](https://mlsanity.com/paper/2311.12812/) (2023)
- [Enhancing Autism Spectrum Disorder Early Detection with the Parent-Child Dyads Block-Play Protocol and an Attention-enhanced GCN-xLSTM Hybrid Deep Learning Framework](https://mlsanity.com/paper/2408.16924/) (2024)
- [A Novel Dataset for Video-Based Neurodivergent Classification Leveraging Extra-Stimulatory Behavior](https://mlsanity.com/paper/2409.04598/) (2024)
- [Advanced Gesture Recognition for Autism Spectrum Disorder Detection: Integrating YOLOv7, Video Augmentation, and VideoMAE for Naturalistic Video Analysis](https://mlsanity.com/paper/2410.09339/) (2024)
- [Summary of the Unusual Activity Recognition Challenge for Developmental Disability Support](https://mlsanity.com/paper/2601.17049/) (2026)
- [Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing](https://mlsanity.com/paper/2609.24791/) (2026)

## Abstract (arXiv)

Activity recognition computer vision algorithms can be used to detect the presence of autism-related behaviors, including what are termed "restricted and repetitive behaviors", or stimming, by diagnostic instruments. The limited data that exist in this domain are usually recorded with a handheld camera which can be shaky or even moving, posing a challenge for traditional feature representation approaches for activity detection which mistakenly capture the camera's motion as a feature. To address these issues, we first document the advantages and limitations of current feature representation techniques for activity recognition when applied to head banging detection. We then propose a feature representation consisting exclusively of head pose keypoints. We create a computer vision classifier for detecting head banging in home videos using a time-distributed convolutional neural network (CNN) in which a single CNN extracts features from each frame in the input sequence, and these extracted features are fed as input to a long short-term memory (LSTM) network. On the binary task of predicting head banging and no head banging within videos from the Self Stimulatory Behaviour Dataset (SSBD), we reach a mean F1-score of 90.77% using 3-fold cross validation (with individual fold F1-scores of 83.3%, 89.0%, and 100.0%) when ensuring that no child who appeared in the train set was in the test set for all folds. This work documents a successful technique for training a computer vision classifier which can detect human motion with few training examples and even when the camera recording the source clips is unstable. The general methods described here can be applied by designers and developers of interactive systems towards other human motion and pose classification problems used in mobile and ubiquitous interactive systems.