# Dual-Modality Vehicle Anomaly Detection via Bilateral Trajectory Tracing

arXiv: https://arxiv.org/abs/2106.05003 · HTML: https://mlsanity.com/paper/2106.05003/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection](https://mlsanity.com/paper/1903.07256/) (2019)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)
- [Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection](https://mlsanity.com/paper/2003.06780/) (2020)
- [Few-shot Scene-adaptive Anomaly Detection](https://mlsanity.com/paper/2007.07843/) (2020)
- [Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge](https://mlsanity.com/paper/1709.09121/) (2017)

## Built on by (most cited first, 4 of 4)

- [Anomaly Detection in Multi-Agent Trajectories for Automated Driving](https://mlsanity.com/paper/2110.07922/) (2021)
- [Detecting Socially Abnormal Highway Driving Behaviors via Recurrent Graph Attention Networks](https://mlsanity.com/paper/2304.11513/) (2023)
- [CosmosDSR -- a methodology for automated detection and tracking of orbital debris using the Unscented Kalman Filter](https://mlsanity.com/paper/2310.17158/) (2023)
- [Improving the Detection of Small Oriented Objects in Aerial Images](https://mlsanity.com/paper/2401.06503/) (2024)

## Abstract (arXiv)

Traffic anomaly detection has played a crucial role in Intelligent Transportation System (ITS). The main challenges of this task lie in the highly diversified anomaly scenes and variational lighting conditions. Although much work has managed to identify the anomaly in homogenous weather and scene, few resolved to cope with complex ones. In this paper, we proposed a dual-modality modularized methodology for the robust detection of abnormal vehicles. We introduced an integrated anomaly detection framework comprising the following modules: background modeling, vehicle tracking with detection, mask construction, Region of Interest (ROI) backtracking, and dual-modality tracing. Concretely, we employed background modeling to filter the motion information and left the static information for later vehicle detection. For the vehicle detection and tracking module, we adopted YOLOv5 and multi-scale tracking to localize the anomalies. Besides, we utilized the frame difference and tracking results to identify the road and obtain the mask. In addition, we introduced multiple similarity estimation metrics to refine the anomaly period via backtracking. Finally, we proposed a dual-modality bilateral tracing module to refine the time further. The experiments conducted on the Track 4 testset of the NVIDIA 2021 AI City Challenge yielded a result of 0.9302 F1-Score and 3.4039 root mean square error (RMSE), indicating the effectiveness of our framework.