# Human-Scene Network: A Novel Baseline with Self-rectifying Loss for Weakly supervised Video Anomaly Detection

arXiv: https://arxiv.org/abs/2301.07923 · HTML: https://mlsanity.com/paper/2301.07923/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 19 Jan 2023.

## Builds on (most cited first, 25 of 25)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [Temporal Convolutional Networks: A Unified Approach to Action Segmentation](https://mlsanity.com/paper/1608.08242/) (2016)
- [Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection](https://mlsanity.com/paper/1903.07256/) (2019)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)
- [Generative Cooperative Learning for Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2203.03962/) (2022)
- [Street Scene: A new dataset and evaluation protocol for video anomaly detection](https://mlsanity.com/paper/1902.05872/) (2019)
- [MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2211.15098/) (2022)
- [GODS: Generalized One-class Discriminative Subspaces for Anomaly Detection](https://mlsanity.com/paper/1908.05884/) (2019)
- [Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection](https://mlsanity.com/paper/1908.04321/) (2019)
- [Localizing Anomalies from Weakly-Labeled Videos](https://mlsanity.com/paper/2008.08944/) (2020)
- [Motion-Aware Feature for Improved Video Anomaly Detection](https://mlsanity.com/paper/1907.10211/) (2019)
- [Weakly Supervised Video Anomaly Detection via Center-guided Discriminative Learning](https://mlsanity.com/paper/2104.07268/) (2021)
- [A Self-Reasoning Framework for Anomaly Detection Using Video-Level Labels](https://mlsanity.com/paper/2008.11887/) (2020)
- [Anomaly Detection in Video Sequences: A Benchmark and Computational Model](https://mlsanity.com/paper/2106.08570/) (2021)
- [Local Anomaly Detection in Videos using Object-Centric Adversarial Learning](https://mlsanity.com/paper/2011.06722/) (2020)
- [Weakly-supervised Joint Anomaly Detection and Classification](https://mlsanity.com/paper/2108.08996/) (2021)
- [3D ResNet with Ranking Loss Function for Abnormal Activity Detection in Videos](https://mlsanity.com/paper/2002.01132/) (2020)

## Built on by (most cited first, 4 of 4)

- [Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2508.06318/) (2025)
- [Just Dance with π! A Poly-modal Inductor for Weakly-supervised Video Anomaly Detection](https://mlsanity.com/paper/2505.13123/) (2025)
- [Multimodal Attention-Enhanced Feature Fusion-based Weekly Supervised Anomaly Violence Detection](https://mlsanity.com/paper/2409.11223/) (2024)
- [A Lightweight Video Anomaly Detection Model with Weak Supervision and Adaptive Instance Selection](https://mlsanity.com/paper/2310.05330/) (2023)

## Abstract (arXiv)

Video anomaly detection in surveillance systems with only video-level labels (i.e. weakly-supervised) is challenging. This is due to, (i) the complex integration of human and scene based anomalies comprising of subtle and sharp spatio-temporal cues in real-world scenarios, (ii) non-optimal optimization between normal and anomaly instances under weak supervision. In this paper, we propose a Human-Scene Network to learn discriminative representations by capturing both subtle and strong cues in a dissociative manner. In addition, a self-rectifying loss is also proposed that dynamically computes the pseudo temporal annotations from video-level labels for optimizing the Human-Scene Network effectively. The proposed Human-Scene Network optimized with self-rectifying loss is validated on three publicly available datasets i.e. UCF-Crime, ShanghaiTech and IITB-Corridor, outperforming recently reported state-of-the-art approaches on five out of the six scenarios considered.