# Event and Activity Recognition in Video Surveillance for Cyber-Physical Systems

arXiv: https://arxiv.org/abs/2111.02064 · HTML: https://mlsanity.com/paper/2111.02064/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 71 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 12 of 12)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [ActionVLAD: Learning spatio-temporal aggregation for action classification](https://mlsanity.com/paper/1704.02895/) (2017)
- [Rosetta: Large scale system for text detection and recognition in images](https://mlsanity.com/paper/1910.05085/) (2019)
- [Attentional Pooling for Action Recognition](https://mlsanity.com/paper/1711.01467/) (2017)
- [A Siamese Long Short-Term Memory Architecture for Human Re-Identification](https://mlsanity.com/paper/1607.08381/) (2016)
- [A Multi-Object Rectified Attention Network for Scene Text Recognition](https://mlsanity.com/paper/1901.03003/) (2019)
- [Learn to Combine Modalities in Multimodal Deep Learning](https://mlsanity.com/paper/1805.11730/) (2018)
- [Fusing Multi-Stream Deep Networks for Video Classification](https://mlsanity.com/paper/1509.06086/) (2015)
- [Second-order Temporal Pooling for Action Recognition](https://mlsanity.com/paper/1704.06925/) (2017)
- [Attention-based Temporal Weighted Convolutional Neural Network for Action Recognition](https://mlsanity.com/paper/1803.07179/) (2018)
- [Human Action Recognition with Deep Temporal Pyramids](https://mlsanity.com/paper/1905.00745/) (2019)

## Built on by (most cited first, 2 of 2)

- [Unsupervised Action Localization Crop in Video Retargeting for 3D ConvNets](https://mlsanity.com/paper/2111.07426/) (2021)
- [Recent Trends in 2D Object Detection and Applications in Video Event Recognition](https://mlsanity.com/paper/2202.03206/) (2022)

## Abstract (arXiv)

This chapter aims to aid the development of Cyber-Physical Systems (CPS) in automated understanding of events and activities in various applications of video-surveillance. These events are mostly captured by drones, CCTVs or novice and unskilled individuals on low-end devices. Being unconstrained, these videos are immensely challenging due to a number of quality factors. We present an extensive account of the various approaches taken to solve the problem over the years. This ranges from methods as early as Structure from Motion (SFM) based approaches to recent solution frameworks involving deep neural networks. We show that the long-term motion patterns alone play a pivotal role in the task of recognizing an event. Consequently each video is significantly represented by a fixed number of key-frames using a graph-based approach. Only the temporal features are exploited using a hybrid Convolutional Neural Network (CNN) + Recurrent Neural Network (RNN) architecture. The results we obtain are encouraging as they outperform standard temporal CNNs and are at par with those using spatial information along with motion cues. Further exploring multistream models, we conceive a multi-tier fusion strategy for the spatial and temporal wings of a network. A consolidated representation of the respective individual prediction vectors on video and frame levels is obtained using a biased conflation technique. The fusion strategy endows us with greater rise in precision on each stage as compared to the state-of-the-art methods, and thus a powerful consensus is achieved in classification. Results are recorded on four benchmark datasets widely used in the domain of action recognition, namely CCV, HMDB, UCF-101 and KCV. It is inferable that focusing on better classification of the video sequences certainly leads to robust actuation of a system designed for event surveillance and object cum activity tracking.