# Spatiotemporal modeling of grip forces captures proficiency in manual robot control

arXiv: https://arxiv.org/abs/2303.01995 · HTML: https://mlsanity.com/paper/2303.01995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 72 references that are in the map.
- First posted on arXiv 3 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Getting nowhere fast: trade-off between speed and precision in training to execute image-guided hand-tool movements](https://mlsanity.com/paper/1803.11280/) (2018)
- [Effects of 2D and 3D image views on hand movement trajectories in the surgeons peripersonal space in a computer controlled simulator environment](https://mlsanity.com/paper/1803.11283/) (2018)
- [Seeing virtual while acting real: Visual display and strategy effects on the time and precision of eye-hand coordination](https://mlsanity.com/paper/1804.00974/) (2018)
- [Towards expert-based speed-precision control in early simulator training for novice surgeons](https://mlsanity.com/paper/1904.06710/) (2019)
- [The quantization error in a Self-Organizing Map as a contrast and colour specific indicator of single-pixel change in large random patterns](https://mlsanity.com/paper/2011.03970/) (2020)
- [Wearable Sensors for Individual Grip Force Profiling](https://mlsanity.com/paper/2011.05863/) (2020)
- [Seven properties of self-organization in the human brain](https://mlsanity.com/paper/2011.05868/) (2020)
- [Sensors for expert grip force profiling: towards benchmarking manual control of a robotic device for surgical tool movements](https://mlsanity.com/paper/2011.06449/) (2020)
- [Correlating grip force signals from multiple sensors highlights prehensile control strategies in a complex task-user system](https://mlsanity.com/paper/2011.06452/) (2020)
- [Principles of perceptual grouping: implications for image-guided surgery](https://mlsanity.com/paper/1808.01640/) (2018)
- [Human Symmetry Uncertainty Detected by a Self-Organizing Neural Network Map](https://mlsanity.com/paper/2103.00256/) (2021)
- [Making Sense of Complex Sensor Data Streams](https://mlsanity.com/paper/2106.09500/) (2021)
- [From Biological Synapses to Intelligent Robots](https://mlsanity.com/paper/2202.12660/) (2022)
- [Grip force as a functional window to somatosensory cognition](https://mlsanity.com/paper/2210.08583/) (2022)

## Abstract (arXiv)

This paper builds on our previous work by exploiting Artificial Intelligence to predict individual grip force variability in manual robot control. Grip forces were recorded from various loci in the dominant and non dominant hands of individuals by means of wearable wireless sensor technology. Statistical analyses bring to the fore skill specific temporal variations in thousands of grip forces of a complete novice and a highly proficient expert in manual robot control. A brain inspired neural network model that uses the output metric of a Self Organizing Map with unsupervised winner take all learning was run on the sensor output from both hands of each user. The neural network metric expresses the difference between an input representation and its model representation at any given moment in time t and reliably captures the differences between novice and expert performance in terms of grip force variability.Functionally motivated spatiotemporal analysis of individual average grip forces, computed for time windows of constant size in the output of a restricted amount of task-relevant sensors in the dominant (preferred) hand, reveal finger-specific synergies reflecting robotic task skill. The analyses lead the way towards grip force monitoring in real time to permit tracking task skill evolution in trainees, or identify individual proficiency levels in human robot interaction in environmental contexts of high sensory uncertainty. Parsimonious Artificial Intelligence (AI) assistance will contribute to the outcome of new types of surgery, in particular single-port approaches such as NOTES (Natural Orifice Transluminal Endoscopic Surgery) and SILS (Single Incision Laparoscopic Surgery).