# High Speed Human Action Recognition using a Photonic Reservoir Computer

arXiv: https://arxiv.org/abs/2305.15283 · HTML: https://mlsanity.com/paper/2305.15283/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 98 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification](https://mlsanity.com/paper/1411.6447/) (2014)
- [Reinforcement Learning in a large scale photonic Recurrent Neural Network](https://mlsanity.com/paper/1711.05133/) (2017)
- [Human action recognition with a large-scale brain-inspired photonic computer](https://mlsanity.com/paper/2004.02545/) (2020)
- [Bayesian optimisation of large-scale photonic reservoir computers](https://mlsanity.com/paper/2004.02535/) (2020)
- [Large-scale spatiotemporal photonic reservoir computer for image classification](https://mlsanity.com/paper/2004.02542/) (2020)
- [Brain-inspired photonic signal processor for periodic pattern generation and chaotic system emulation](https://mlsanity.com/paper/1802.02026/) (2018)

## Built on by (most cited first, 5 of 5)

- [Efficient Optimisation of Physical Reservoir Computers using only a Delayed Input](https://mlsanity.com/paper/2401.14371/) (2024)
- [TransNet: A Transfer Learning-Based Network for Human Action Recognition](https://mlsanity.com/paper/2309.06951/) (2023)
- [Evading Detection Actively: Toward Anti-Forensics against Forgery Localization](https://mlsanity.com/paper/2310.10036/) (2023)
- [Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition](https://mlsanity.com/paper/2505.22985/) (2025)
- [On the Role of Preprocessing and Memristor Dynamics in Reservoir Computing for Image Classification](https://mlsanity.com/paper/2604.21602/) (2026)

## Abstract (arXiv)

The recognition of human actions in videos is one of the most active research fields in computer vision. The canonical approach consists in a more or less complex preprocessing stages of the raw video data, followed by a relatively simple classification algorithm. Here we address recognition of human actions using the reservoir computing algorithm, which allows us to focus on the classifier stage. We introduce a new training method for the reservoir computer, based on "Timesteps Of Interest", which combines in a simple way short and long time scales. We study the performance of this algorithm using both numerical simulations and a photonic implementation based on a single non-linear node and a delay line on the well known KTH dataset. We solve the task with high accuracy and speed, to the point of allowing for processing multiple video streams in real time. The present work is thus an important step towards developing efficient dedicated hardware for video processing.