# Hypergraph-based Multi-View Action Recognition using Event Cameras

arXiv: https://arxiv.org/abs/2403.19316 · HTML: https://mlsanity.com/paper/2403.19316/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 97 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Graph Clustering with Graph Neural Networks](https://mlsanity.com/paper/2006.16904/) (2020)
- [G-Mixup: Graph Data Augmentation for Graph Classification](https://mlsanity.com/paper/2202.07179/) (2022)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [An Exponential Learning Rate Schedule for Deep Learning](https://mlsanity.com/paper/1910.07454/) (2019)
- [Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition](https://mlsanity.com/paper/1711.11152/) (2017)
- [Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors](https://mlsanity.com/paper/1505.04868/) (2015)
- [Temporal Binary Representation for Event-Based Action Recognition](https://mlsanity.com/paper/2010.08946/) (2020)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [Human Action Recognition using Factorized Spatio-Temporal Convolutional Networks](https://mlsanity.com/paper/1510.00562/) (2015)

## Built on by (most cited first, 19 of 19)

- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithms](https://mlsanity.com/paper/2408.09764/) (2024)
- [MTevent: A Multi-Task Event Camera Dataset for 6D Pose Estimation and Moving Object Detection](https://mlsanity.com/paper/2505.11282/) (2025)
- [CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework](https://mlsanity.com/paper/2504.12576/) (2025)
- [MoCom: Motion-based Inter-MAV Visual Communication Using Event Vision and Spiking Neural Networks](https://mlsanity.com/paper/2510.14770/) (2025)
- [Event USKT : U-State Space Model in Knowledge Transfer for Event Cameras](https://mlsanity.com/paper/2411.15276/) (2024)
- [Event-based Solutions for Human-centered Applications: A Comprehensive Review](https://mlsanity.com/paper/2502.18490/) (2025)
- [PASS: Path-selective State Space Model for Event-based Recognition](https://mlsanity.com/paper/2409.16953/) (2024)
- [Low-Latency Scalable Streaming for Event-Based Vision](https://mlsanity.com/paper/2412.07889/) (2024)
- [Evolving Skeletons: Motion Dynamics in Action Recognition](https://mlsanity.com/paper/2501.02593/) (2025)
- [EHGCN: Hierarchical Euclidean-Hyperbolic Fusion via Motion-Aware GCN for Hybrid Event Stream Perception](https://mlsanity.com/paper/2504.16616/) (2025)
- [Multi Camera Connected Vision System with Multi View Analytics: A Comprehensive Survey](https://mlsanity.com/paper/2510.09731/) (2025)
- [SMV-EAR: Bring Spatiotemporal Multi-View Representation Learning into Efficient Event-Based Action Recognition](https://mlsanity.com/paper/2601.17391/) (2026)
- [Multi-Modal Parameter-Efficient Fine-tuning via Graph Neural Network](https://mlsanity.com/paper/2408.00290/) (2024)
- [RGB-Event HyperGraph Prompt for Kilometer Marker Recognition based on Pre-trained Foundation Models](https://mlsanity.com/paper/2602.22026/) (2026)
- [NeuroLip: An Event-driven Spatiotemporal Learning Framework for Cross-Scene Lip-Motion-based Visual Speaker Recognition](https://mlsanity.com/paper/2604.15718/) (2026)
- [FLASH: Efficient Impact Fall Detection with Unified Hypergraph State-Space Model](https://mlsanity.com/paper/2607.25791/) (2026)
- [Event ActivityNet: A Large-Scale Simulated-Event Benchmark for Untrimmed Action Understanding](https://mlsanity.com/paper/2608.01948/) (2026)
- [Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchen](https://mlsanity.com/paper/2608.04865/) (2026)

## Abstract (arXiv)

Action recognition from video data forms a cornerstone with wide-ranging applications. Single-view action recognition faces limitations due to its reliance on a single viewpoint. In contrast, multi-view approaches capture complementary information from various viewpoints for improved accuracy. Recently, event cameras have emerged as innovative bio-inspired sensors, leading to advancements in event-based action recognition. However, existing works predominantly focus on single-view scenarios, leaving a gap in multi-view event data exploitation, particularly in challenges like information deficit and semantic misalignment. To bridge this gap, we introduce HyperMV, a multi-view event-based action recognition framework. HyperMV converts discrete event data into frame-like representations and extracts view-related features using a shared convolutional network. By treating segments as vertices and constructing hyperedges using rule-based and KNN-based strategies, a multi-view hypergraph neural network that captures relationships across viewpoint and temporal features is established. The vertex attention hypergraph propagation is also introduced for enhanced feature fusion. To prompt research in this area, we present the largest multi-view event-based action dataset THUMV-EACT-50, comprising 50 actions from 6 viewpoints, which surpasses existing datasets by over tenfold. Experimental results show that HyperMV significantly outperforms baselines in both cross-subject and cross-view scenarios, and also exceeds the state-of-the-arts in frame-based multi-view action recognition.