# End-to-End Deep Learning for Real-Time Neuroimaging-Based Assessment of Bimanual Motor Skills

arXiv: https://arxiv.org/abs/2504.03681 · HTML: https://mlsanity.com/paper/2504.03681/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 87 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Video-based surgical skill assessment using 3D convolutional neural networks](https://mlsanity.com/paper/1903.02306/) (2019)
- [Towards Unified Surgical Skill Assessment](https://mlsanity.com/paper/2106.01035/) (2021)
- [How Much Can We Really Trust You? Towards Simple, Interpretable Trust Quantification Metrics for Deep Neural Networks](https://mlsanity.com/paper/2009.05835/) (2020)
- [Understanding Geometry of Encoder-Decoder CNNs](https://mlsanity.com/paper/1901.07647/) (2019)
- [Deep Learning in fNIRS: A review](https://mlsanity.com/paper/2201.13371/) (2022)
- [Deep Neural Networks for the Assessment of Surgical Skills: A Systematic Review](https://mlsanity.com/paper/2103.05113/) (2021)
- [Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning](https://mlsanity.com/paper/2203.09589/) (2022)
- [A deep learning model for burn depth classification using ultrasound imaging](https://mlsanity.com/paper/2203.15879/) (2022)

## Built on by (most cited first, 1 of 1)

- [An Interpretable Transformer-Based Foundation Model for Cross-Procedural Skill Assessment Using Raw fNIRS Signals](https://mlsanity.com/paper/2506.22476/) (2025)

## Abstract (arXiv)

The real-time assessment of complex motor skills presents a challenge in fields such as surgical training and rehabilitation. Recent advancements in neuroimaging, particularly functional near-infrared spectroscopy (fNIRS), have enabled objective assessment of such skills with high accuracy. However, these techniques are hindered by extensive preprocessing requirements to extract neural biomarkers. This study presents a novel end-to-end deep learning framework that processes raw fNIRS signals directly, eliminating the need for intermediate preprocessing steps. The model was evaluated on datasets from three distinct bimanual motor tasks--suturing, pattern cutting, and endotracheal intubation (ETI)--using performance metrics derived from both training and retention datasets. It achieved a mean classification accuracy of 93.9% (SD 4.4) and a generalization accuracy of 92.6% (SD 1.9) on unseen skill retention datasets, with a leave-one-subject-out cross-validation yielding an accuracy of 94.1% (SD 3.6). Contralateral prefrontal cortex activations exhibited task-specific discriminative power, while motor cortex activations consistently contributed to accurate classification. The model also demonstrated resilience to neurovascular coupling saturation caused by extended task sessions, maintaining robust performance across trials. Comparative analysis confirms that the end-to-end model performs on par with or surpasses baseline models optimized for fully processed fNIRS data, with statistically similar (p<0.05) or improved prediction accuracies. By eliminating the need for extensive signal preprocessing, this work provides a foundation for real-time, non-invasive assessment of bimanual motor skills in medical training environments, with potential applications in robotics, rehabilitation, and sports.