# Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM

arXiv: https://arxiv.org/abs/2102.10590 · HTML: https://mlsanity.com/paper/2102.10590/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 21 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [RWF-2000: An Open Large Scale Video Database for Violence Detection](https://mlsanity.com/paper/1911.05913/) (2019)
- [Learning to Detect Violent Videos using Convolutional Long Short-Term Memory](https://mlsanity.com/paper/1709.06531/) (2017)
- [Separable Convolutional LSTMs for Faster Video Segmentation](https://mlsanity.com/paper/1907.06876/) (2019)

## Built on by (most cited first, 12 of 12)

- [Prompt-Guided Zero-Shot Anomaly Action Recognition using Pretrained Deep Skeleton Features](https://mlsanity.com/paper/2303.15167/) (2023)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [SSIVD-Net: A Novel Salient Super Image Classification & Detection Technique for Weaponized Violence](https://mlsanity.com/paper/2207.12850/) (2022)
- [Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures](https://mlsanity.com/paper/2308.05106/) (2023)
- [JOSENet: A Joint Stream Embedding Network for Violence Detection in Surveillance Videos](https://mlsanity.com/paper/2405.02961/) (2024)
- [Reinforcement Learning-based Mixture of Vision Transformers for Video Violence Recognition](https://mlsanity.com/paper/2310.03108/) (2023)
- [DIFEM: Key-points Interaction based Feature Extraction Module for Violence Recognition in Videos](https://mlsanity.com/paper/2412.05386/) (2024)
- [Vi-SAFE: A Spatial-Temporal Framework for Efficient Violence Detection in Public Surveillance](https://mlsanity.com/paper/2509.13210/) (2025)
- [Short-Window Sliding Learning for Real-Time Violence Detection via LLM-based Auto-Labeling](https://mlsanity.com/paper/2511.10866/) (2025)
- [Contrastive learning-based video quality assessment-jointed video vision transformer for video recognition](https://mlsanity.com/paper/2603.10965/) (2026)
- [Dual-Temporal LSTM with Hybrid Attention for Airline Passenger Load Factor Forecasting: Integrating Intra-Flight and Inter-Flight Booking Dynamics](https://mlsanity.com/paper/2605.11569/) (2026)
- [Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining](https://mlsanity.com/paper/2608.25701/) (2026)

## Abstract (arXiv)

Automatically detecting violence from surveillance footage is a subset of activity recognition that deserves special attention because of its wide applicability in unmanned security monitoring systems, internet video filtration, etc. In this work, we propose an efficient two-stream deep learning architecture leveraging Separable Convolutional LSTM (SepConvLSTM) and pre-trained MobileNet where one stream takes in background suppressed frames as inputs and other stream processes difference of adjacent frames. We employed simple and fast input pre-processing techniques that highlight the moving objects in the frames by suppressing non-moving backgrounds and capture the motion in-between frames. As violent actions are mostly characterized by body movements these inputs help produce discriminative features. SepConvLSTM is constructed by replacing convolution operation at each gate of ConvLSTM with a depthwise separable convolution that enables producing robust long-range Spatio-temporal features while using substantially fewer parameters. We experimented with three fusion methods to combine the output feature maps of the two streams. Evaluation of the proposed methods was done on three standard public datasets. Our model outperforms the accuracy on the larger and more challenging RWF-2000 dataset by more than a 2% margin while matching state-of-the-art results on the smaller datasets. Our experiments lead us to conclude, the proposed models are superior in terms of both computational efficiency and detection accuracy.