# Online Recognition of Actions Involving Objects

arXiv: https://arxiv.org/abs/2104.06070 · HTML: https://mlsanity.com/paper/2104.06070/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 44 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Built on by (most cited first, 5 of 5)

- [Hierarchical growing grid networks for skeleton based action recognition](https://mlsanity.com/paper/2104.11165/) (2021)
- [Online recognition of unsegmented actions with hierarchical SOM architecture](https://mlsanity.com/paper/2104.11637/) (2021)
- [Action in Mind: A Neural Network Approach to Action Recognition and Segmentation](https://mlsanity.com/paper/2104.14870/) (2021)
- [Video Relationship Detection Using Mixture of Experts](https://mlsanity.com/paper/2403.03994/) (2024)
- [Predicting the intended action using internal simulation of perception](https://mlsanity.com/paper/2202.04466/) (2022)

## Abstract (arXiv)

We present an online system for real time recognition of actions involving objects working in online mode. The system merges two streams of information processing running in parallel. One is carried out by a hierarchical self-organizing map (SOM) system that recognizes the performed actions by analysing the spatial trajectories of the agent's movements. It consists of two layers of SOMs and a custom made supervised neural network. The activation sequences in the first layer SOM represent the sequences of significant postures of the agent during the performance of actions. These activation sequences are subsequently recoded and clustered in the second layer SOM, and then labeled by the activity in the third layer custom made supervised neural network. The second information processing stream is carried out by a second system that determines which object among several in the agent's vicinity the action is applied to. This is achieved by applying a proximity measure. The presented method combines the two information processing streams to determine what action the agent performed and on what object. The action recognition system has been tested with excellent performance.