# Online recognition of unsegmented actions with hierarchical SOM architecture

arXiv: https://arxiv.org/abs/2104.11637 · HTML: https://mlsanity.com/paper/2104.11637/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 55 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [Learning to track for spatio-temporal action localization](https://mlsanity.com/paper/1506.01929/) (2015)
- [Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks](https://mlsanity.com/paper/1604.05633/) (2016)
- [Predictive-Corrective Networks for Action Detection](https://mlsanity.com/paper/1704.03615/) (2017)
- [First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition](https://mlsanity.com/paper/2104.06059/) (2021)
- [Online Recognition of Actions Involving Objects](https://mlsanity.com/paper/2104.06070/) (2021)
- [Hierarchical growing grid networks for skeleton based action recognition](https://mlsanity.com/paper/2104.11165/) (2021)

## Built on by (most cited first, 3 of 3)

- [Action in Mind: A Neural Network Approach to Action Recognition and Segmentation](https://mlsanity.com/paper/2104.14870/) (2021)
- [Predictive Event Segmentation and Representation with Neural Networks: A Self-Supervised Model Assessed by Psychological Experiments](https://mlsanity.com/paper/2210.05710/) (2022)
- [Predicting the intended action using internal simulation of perception](https://mlsanity.com/paper/2202.04466/) (2022)

## Abstract (arXiv)

Automatic recognition of an online series of unsegmented actions requires a method for segmentation that determines when an action starts and when it ends. In this paper, a novel approach for recognizing unsegmented actions in online test experiments is proposed. The method uses self-organizing neural networks to build a three-layer cognitive architecture. The unique features of an action sequence are represented as a series of elicited key activations by the first-layer self-organizing map. An average length of a key activation vector is calculated for all action sequences in a training set and adjusted in learning trials to generate input patterns to the second-layer self-organizing map. The pattern vectors are clustered in the second layer, and the clusters are then labeled by an action identity in the third layer neural network. The experiment results show that although the performance drops slightly in online experiments compared to the offline tests, the ability of the proposed architecture to deal with the unsegmented action sequences as well as the online performance makes the system more plausible and practical in real-case scenarios.