# Human action recognition with a large-scale brain-inspired photonic computer

arXiv: https://arxiv.org/abs/2004.02545 · HTML: https://mlsanity.com/paper/2004.02545/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 15 of 15)

- [All-optical graph representation learning using integrated diffractive photonic computing units](https://mlsanity.com/paper/2204.10978/) (2022)
- [Parallel photonic accelerator for decision making using optical spatiotemporal chaos](https://mlsanity.com/paper/2210.06976/) (2022)
- [Persistent Homology Meets Object Unity: Object Recognition in Clutter](https://mlsanity.com/paper/2305.03815/) (2023)
- [High Speed Human Action Recognition using a Photonic Reservoir Computer](https://mlsanity.com/paper/2305.15283/) (2023)
- [Master memory function for delay-based reservoir computers with single-variable dynamics](https://mlsanity.com/paper/2108.12643/) (2021)
- [Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework](https://mlsanity.com/paper/2110.00214/) (2021)
- [Efficient Optimisation of Physical Reservoir Computers using only a Delayed Input](https://mlsanity.com/paper/2401.14371/) (2024)
- [EEG Opto-processor: epileptic seizure detection using diffractive photonic computing units](https://mlsanity.com/paper/2301.10167/) (2022)
- [A differentiable brain simulator bridging brain simulation and brain-inspired computing](https://mlsanity.com/paper/2311.05106/) (2023)
- [Efficient Human Vision Inspired Action Recognition using Adaptive Spatiotemporal Sampling](https://mlsanity.com/paper/2207.05249/) (2022)
- [Advanced Computing and Related Applications Leveraging Brain-inspired Spiking Neural Networks](https://mlsanity.com/paper/2309.04426/) (2023)
- [Human-Inspired Topological Representations for Visual Object Recognition in Unseen Environments](https://mlsanity.com/paper/2309.08239/) (2023)
- [Artificial intelligence optical hardware empowers high-resolution hyperspectral video understanding at 1.2 Tb/s](https://mlsanity.com/paper/2312.10639/) (2023)
- [THOR2: Topological Analysis for 3D Shape and Color-Based Human-Inspired Object Recognition in Unseen Environments](https://mlsanity.com/paper/2408.01579/) (2024)
- [Photonic reservoir computing with complex networks](https://mlsanity.com/paper/2607.23285/) (2026)

## Abstract (arXiv)

The recognition of human actions in video streams is a challenging task in computer vision, with cardinal applications in e.g. brain-computer interface and surveillance. Deep learning has shown remarkable results recently, but can be found hard to use in practice, as its training requires large datasets and special purpose, energy-consuming hardware. In this work, we propose a scalable photonic neuro-inspired architecture based on the reservoir computing paradigm, capable of recognising video-based human actions with state-of-the-art accuracy. Our experimental optical setup comprises off-the-shelf components, and implements a large parallel recurrent neural network that is easy to train and can be scaled up to hundreds of thousands of nodes. This work paves the way towards simply reconfigurable and energy-efficient photonic information processing systems for real-time video processing.