# Deep learning empowered sensor fusion boosts infant movement classification

arXiv: https://arxiv.org/abs/2406.09014 · HTML: https://mlsanity.com/paper/2406.09014/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 68 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [A Spatio-temporal Attention-based Model for Infant Movement Assessment from Videos](https://mlsanity.com/paper/2105.09783/) (2021)
- [Automatic Posture and Movement Tracking of Infants with Wearable Movement Sensors](https://mlsanity.com/paper/1909.09823/) (2019)
- [Towards Explainable Abnormal Infant Movements Identification: A Body-part Based Prediction and Visualisation Framework](https://mlsanity.com/paper/2106.04966/) (2021)
- [Open video data sharing in developmental and behavioural science](https://mlsanity.com/paper/2207.11020/) (2022)
- [Mobile solutions for clinical surveillance and evaluation in infancy -- General Movement Apps](https://mlsanity.com/paper/2303.14699/) (2023)
- [Infant movement classification through pressure distribution analysis](https://mlsanity.com/paper/2208.00884/) (2022)

## Built on by (most cited first, 4 of 4)

- [Comparison of marker-less 2D image-based methods for infant pose estimation](https://mlsanity.com/paper/2410.04980/) (2024)
- [Deep Temporal Modeling and Ensemble Fusion for Multimodal Emotion Recognition from Physiological Signals](https://mlsanity.com/paper/2606.15026/) (2026)
- [Assessing Low Back Movement with Motion Tape Sensor Data Through Deep Learning](https://mlsanity.com/paper/2602.11465/) (2026)
- [Uncertainty-Aware Federated Learning for Infant Movement Analysis](https://mlsanity.com/paper/2609.31463/) (2026)

## Abstract (arXiv)

To assess the integrity of the developing nervous system, the Prechtl general movement assessment (GMA) is recognized for its clinical value in diagnosing neurological impairments in early infancy. GMA has been increasingly augmented through machine learning approaches intending to scale-up its application, circumvent costs in the training of human assessors and further standardize classification of spontaneous motor patterns. Available deep learning tools, all of which are based on single sensor modalities, are however still considerably inferior to that of well-trained human assessors. These approaches are hardly comparable as all models are designed, trained and evaluated on proprietary/silo-data sets. With this study we propose a sensor fusion approach for assessing fidgety movements (FMs). FMs were recorded from 51 typically developing participants. We compared three different sensor modalities (pressure, inertial, and visual sensors). Various combinations and two sensor fusion approaches (late and early fusion) for infant movement classification were tested to evaluate whether a multi-sensor system outperforms single modality assessments. Convolutional neural network (CNN) architectures were used to classify movement patterns. The performance of the three-sensor fusion (classification accuracy of 94.5%) was significantly higher than that of any single modality evaluated. We show that the sensor fusion approach is a promising avenue for automated classification of infant motor patterns. The development of a robust sensor fusion system may significantly enhance AI-based early recognition of neurofunctions, ultimately facilitating automated early detection of neurodevelopmental conditions.