# Classification of Abnormal Hand Movement for Aiding in Autism Detection: Machine Learning Study

arXiv: https://arxiv.org/abs/2108.07917 · HTML: https://mlsanity.com/paper/2108.07917/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 69 references that are in the map.
- First posted on arXiv 18 Aug 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Detect-and-Track: Efficient Pose Estimation in Videos](https://mlsanity.com/paper/1712.09184/) (2017)
- [Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbanging](https://mlsanity.com/paper/2101.03478/) (2021)
- [Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems](https://mlsanity.com/paper/2002.06581/) (2020)
- [Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study](https://mlsanity.com/paper/2012.08678/) (2020)
- [Dynamic Control Flow in Large-Scale Machine Learning](https://mlsanity.com/paper/1805.01772/) (2018)
- [Training Affective Computer Vision Models by Crowdsourcing Soft-Target Labels](https://mlsanity.com/paper/2101.03477/) (2021)
- [Machine learning approach for early detection of autism by combining questionnaire and home video screening](https://mlsanity.com/paper/1703.06076/) (2017)
- [A Facial Affect Analysis System for Autism Spectrum Disorder](https://mlsanity.com/paper/1904.03616/) (2019)

## Built on by (most cited first, 10 of 10)

- [Computer Vision Estimation of Emotion Reaction Intensity in the Wild](https://mlsanity.com/paper/2303.10741/) (2023)
- [Video-based Analysis Reveals Atypical Social Gaze in People with Autism Spectrum Disorder](https://mlsanity.com/paper/2409.00664/) (2024)
- [Unsupervised Video Anomaly Detection for Stereotypical Behaviours in Autism](https://mlsanity.com/paper/2302.13748/) (2023)
- [Ensemble Modeling of Multiple Physical Indicators to Dynamically Phenotype Autism Spectrum Disorder](https://mlsanity.com/paper/2408.13255/) (2024)
- [Exploring Speech Pattern Disorders in Autism using Machine Learning](https://mlsanity.com/paper/2405.05126/) (2024)
- [An Exploration of Active Learning for Affective Digital Phenotyping](https://mlsanity.com/paper/2204.01915/) (2022)
- [Personalization of Affective Models to Enable Neuropsychiatric Digital Precision Health Interventions: A Feasibility Study](https://mlsanity.com/paper/2311.12812/) (2023)
- [Advanced Gesture Recognition for Autism Spectrum Disorder Detection: Integrating YOLOv7, Video Augmentation, and VideoMAE for Naturalistic Video Analysis](https://mlsanity.com/paper/2410.09339/) (2024)
- [Challenges in the Differential Classification of Individual Diagnoses from Co-Occurring Autism and ADHD Using Survey Data](https://mlsanity.com/paper/2411.10479/) (2024)
- [Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies](https://mlsanity.com/paper/2607.07957/) (2026)

## Abstract (arXiv)

A formal autism diagnosis can be an inefficient and lengthy process. Families may wait months or longer before receiving a diagnosis for their child despite evidence that earlier intervention leads to better treatment outcomes. Digital technologies which detect the presence of behaviors related to autism can scale access to pediatric diagnoses. This work aims to demonstrate the feasibility of deep learning technologies for detecting hand flapping from unstructured home videos as a first step towards validating whether models and digital technologies can be leveraged to aid with autism diagnoses. We used the Self-Stimulatory Behavior Dataset (SSBD), which contains 75 videos of hand flapping, head banging, and spinning exhibited by children. From all the hand flapping videos, we extracted 100 positive and control videos of hand flapping, each between 2 to 5 seconds in duration. Utilizing both landmark-driven-approaches and MobileNet V2's pretrained convolutional layers, our highest performing model achieved a testing F1 score of 84% (90% precision and 80% recall) when evaluating with 5-fold cross validation 100 times. This work provides the first step towards developing precise deep learning methods for activity detection of autism-related behaviors.