# First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition

arXiv: https://arxiv.org/abs/2104.06059 · HTML: https://mlsanity.com/paper/2104.06059/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 45 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Built on by (most cited first, 5 of 5)

- [Hierarchical growing grid networks for skeleton based action recognition](https://mlsanity.com/paper/2104.11165/) (2021)
- [Online recognition of unsegmented actions with hierarchical SOM architecture](https://mlsanity.com/paper/2104.11637/) (2021)
- [Action in Mind: A Neural Network Approach to Action Recognition and Segmentation](https://mlsanity.com/paper/2104.14870/) (2021)
- [Video Relationship Detection Using Mixture of Experts](https://mlsanity.com/paper/2403.03994/) (2024)
- [Predicting the intended action using internal simulation of perception](https://mlsanity.com/paper/2202.04466/) (2022)

## Abstract (arXiv)

Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available data sets, and compare its performance to the performance with less sophisticated preprocessing of the input. The results show that including the dynamics of the actions improves the performance. We also apply an attention mechanism that focuses on the parts of the body that are the most involved in performing the actions.